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MEMORANDUM  FOR  UNDER  SECRETARY  OF  DEFENSE  FOR  ACQUISITION 

AND  TECHNOLOGY 

ASSISTANT  SECRETARY  OF  DEFENSE  (COMMAND, 
CONTROL,  COMMUNICATIONS  AND 
INTELLIGENCE) 

DIRECTOR,  DEFENSE  MAPPING  AGENCY 

SUBJECT:  Audit  Report  on  Management  of  the  Digital  Production  System 

Development  at  the  Defense  Mapping  Agency  (Report  No.  95-043) 


We  are  providing  this  report  for  your  information  and  use.  The  audit  was  made 
in  response  to  two  anonymous  DoD  Hotline  complaints  regarding  funding,  contracting, 
development,  and  operation  of  projects  at  the  Defense  Mapping  Agency.  The  report 
focuses  on  issues  concerning  the  development  and  oversight  of  the  Digital  Production 
System,  which  provides  the  capability  to  use  digital  imagery  and  computer-assisted 
techniques  to  produce  mapping,  charting,  and  geodesy  products.  Management 
comments  on  a  draft  of  this  report  were  considered  in  preparing  the  final  report. 

Management  comments  were  responsive  to  the  intent  of  the  recommendations; 
however,  the  Assistant  Secretary  of  Defense  (Command,  Control,  Communications  and 
Intelligence)  disagreed  on  the  materiality  of  the  internal  control  weaknesses.  The 
apparent  disagreement  may  be  due  to  semantics,  in  that  management  concurs  that 
serious  weaknesses  were  identified  and  has  taken  corrective  action.  Therefore  no 
response  to  the  final  report  is  required. 

We  appreciate  the  courtesies  extended  to  the  audit  staff.  If  you  have  questions 
on  this  audit,  please  contact  Mr.  Charles  Santoni,  Audit  Program  Director,  at 
(703)  604-9556  (DSN  664-9556)  or  Mr.  John  Mundell,  Audit  Project  Manager,  at 
(703)  604-9562  (DSN  664-9562).  The  distribution  of  this  report  is  listed  in 
Appendix  1.  The  audit  team  members  are  listed  inside  the  back  cover. 

Robert  J.  Lieberman 
Assistant  Inspector  General 
for  Auditing 
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MANAGEMENT  OF  THE  DIGITAL  PRODUCTION 
SYSTEM  DEVELOPMENT  AT  THE 
DEFENSE  MAPPING  AGENCY 


EXECUTIVE  SUMMARY 


Introduction.  We  made  this  audit  in  response  to  two  anonymous  DoD  Hotline 
complaints  containing  25  allegations  (the  audit  reviewed  20  allegations;  other  DoD 
offices  reviewed  5  allegations)  regarding  the  management  of  the  Defense  Mapping 
Agency  (DMA).  DMA's  mission  is  to  produce  and  distribute  mapping,  charting,  and 
geodesy  products  to  all  DoD  Components.  In  1982,  DMA  started  developing  the 
Digital  Production  System  to  produce  mapping,  charting,  and  geodetic  products  using 
digital  imagery  WiA  the  Digital  Production  System,  DMA's  primary  source  material 
is  digital  data  instead  of  film.  Development  and  acquisition  costs  of  the  Digital 
Production  System  totaled  $2.6  billion  in  November  1992.  In  addition,  from  FY  1989 
through  FY  1993,  DMA  spent  $322  million  to  operate  and  maintain  Digital  Production 
System  hardware  and  software  and  $113  million  to  develop  and  procure  Digital 
Production  System  modifications  and  enhancements.  DMA  plans  to  spend  $1.0  billion 
from  FY  1994  through  FY  1999  to  operate  and  maintain  the  Digital  Production  System 
and  to  develop  products  and  capabilities  that  may  directly  or  indirectly  affect  the 
Digital  Production  System. 

Objective.  The  objective  of  the  audit  was  to  evaluate  the  validity  of  the  Hotline 
allegations  that  sound  management  practices  were  not  used  in  the  acquisition  and 
operation  of  DMA  projects.  Specifically,  we  evaluated  the  policies  and  procedures 
used  to  manage  and  review  the  development  of  the  Digital  Production  System. 

Audit  Results.  The  audit  substantiated  or  partly  substantiated  six  allegations  that 
identified  problems  regarding  the  contracting,  development,  and  operation  of  the  DPS. 
The  audit  results  regarding  five  of  those  six  allegations  are  discussed  in  the  findings. 
The  sixth  allegation  is  discussed  in  our  audit  Report  No.  94-076,  "Contracting 
Authority  at  the  Defense  Mapping  Agency,"  April  1,  1994.  In  addition,  the  audit 
substantiated  or  partly  substantiated  10  allegations,  but  identified  no  adverse  effect. 
Four  allegations  were  not  substantiated.  The  audit  results  regarding  the  20  allegations 
are  summarized  in  Appendix  D. 

The  DMA  did  not  identify  customer  requirements  for  the  Flight  Information 
Publications  and  Automated  Air  Facilities  Information  File  products,  analyze  causes  of 
software  problems,  correct  configuration  management  deficiencies,  and  define 
modifications  for  the  Digital  Production  System  finding  A).  Further,  oversight  of  the 
Digital  Production  System  development  and  its  modification  needs  strengthening 
(Finding  B).  Without  improved  development  procedures  and  oversight,  the  DMA  may 
experience  further  delays  in  completing  products  and  expend  unnecessary  resources  for 
modifications. 


Internal  Controls.  The  audit  limited  the  evaluation  of  internal  controls  to  the  DoD 
Hotline  allegations.  The  audit  identified  material  internal  control  weaknesses  in  the 
Defense  Mapping  Agency's  procedures  for  managing  the  development  of  the  Digital 
Production  System.  The  controls  assessed  are  described  in  Part  I  of  the  report,  and  the 
weaknesses  are  discussed  in  Finding  A. 

Potential  Benefits  of  Audit.  Implementation  of  the  recommendations  will  improve  the 
management  and  oversight  of  the  Digital  Production  System.  In  addition,  the 
development  status  of  the  Digital  Production  System  and  the  planned  modifications  to 
the  Digital  Production  System  will  be  determined,  documented,  and  reported  (see 
Appendix  G).  No  monetary  benefits  are  associated  with  this  audit. 

■Summary  of  Recommendations.  We  recommend  procedures  be  implemented  for 
product  specification  development  and  revised  for  problem  reporting.  Also,  we 
recommend  plans  be  developed  for  correcting  configuration  management  deficiencies 
and  modifying  the  Digital  Production  System.  In  addition,  we  recommend  a 
Milestone  IV  (Major  Modification  Approval)  review  of  the  Digital  Production  System 
be  performed  and  independent  oversight  of  mapping,  charting,  and  geodesy  matters  be 
provided. 

Management  Comments.  The  Assistant  Secretary  of  Defense  (Command,  Control, 
Communications  and  Intelligence)  submitted  consolidated  comments  for  himself;  the 
Under  Secretary  of  Defense  for  Acquisition  and  Technology;  and  the  Director,  DMA. 
The  Assistant  Secretary  stated  that  DMA  either  had  implemented  or  was  in  the  process 
of  implementing  revised  procedures  for  product  specification  development,  problem 
reporting,  and  configuration  management.  The  Assistant  Secretary  also  stated  that 
DMA  would  prepare  a  comprehensive  strategic  plan  by  February  1995  to  reduce  DPS 
operation  and  maintenance  costs  and  to  meet  new  requirements  caused  by  the  new 
world  order.  The  Assistant  Secretary  disagreed  that  a  Milestone  IV  review  should  be 
conducted  because  the  planned  modifications  are  below  the  monetary  threshold  for  a 
Milestone  rV  review.  However,  he  stated  that  the  Deputy  Assistant  Secretary  of 
Defense  (Command,  Control,  Communications  and  Intelligence  Acquisition)  would 
conduct  an  in-process  review  of  the  DPS.  The  purpose  of  the  in-process  review  would 
be  to  assess  the  status  of  the  DPS,  to  review  DMA's  strategic  plan  for  the  DPS,  and  to 
determine  intervals  for  future  in-process  reviews.  The  Assistant  Secretary  disagreed 
that  the  internal  control  weaknesses  were  material  because  DMA’s  internal  control 
processes  had  identified  the  problems.  DMA  either  had  already  implemented  or  was  in 
the  process  of  implementing  solutions  to  the  weaknesses.  See  Part  II  for  a  discussion 
of  management's  response  and  Part  IV  for  the  complete  text  of  the  comments. 

Audit  Response.  Management  comments  were  responsive  to  the  recommendations. 
The  corrective  actions  taken  or  planned  to  be  taken  will  resolve  the  internal  control 
weaknesses  discussed  in  the  report  and  improve  management  of  this  large  and 
expensive  program. 
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Introduction 


Background 


Defense  Mapping  Agency  Mission.  The  mission  of  the  Defense  Mapping 
Agency  (DMA)  is  to  provide  support  to  the  Office  of  the  Secretary  of  Defense, 
Military  Departments,  Joint  Staff,  unified  conunands.  Defense  agencies,  and 
other  Federal  Departments  and  Agencies  on  matters  concerning  mapping, 
charting,  and  geodesy  (MC&G).  Also,  DMA  provides  nautical  data  to 
worldwide  merchant  marine  and  private  vessel  operators.  DMA  provides  more 
than  270  types  of  MC&G  products  and  services  to  the  customers  it  supports. 

Purpose  of  the  Digital  Production  System.  DMA  developed  the  Digital 
Production  System  (DPS)  to  provide  a  capability  to  produce  certain  MC&G 
products  from  advanced  imaging  systems  using  digital  imagery  and  computer- 
assisted  techniques.  In  1982,  Congress  endorsed  guidance  by  the  Office  of 
Secretary  of  Defense  that  required  DMA  to  develop  an  all-digital  production 
system  capable  of  meeting  critical  demands  for  MC&G  support.  With  the  new 
production  system,  DMA’s  primary  source  material  is  digital  data  instead  of 
film.  The  DPS  interfaces  with  external  agencies  to  obtain  imagery  and  once  all 
required  imagery  is  collected,  produces  MC&G  products  through  a  tightly 
controlled  and  structured  process.  The  DPS  is  a  DMA-standard  system  used  by 
the  three  DMA  production  components:  Aerospace  Center,  St.  Louis,  Missouri; 
Hydrographic/Topographic  Center,  Brookmont,  Maryland;  and  Reston  Center, 
Reston,  Virginia.  DMA  estimated  that  using  digital  imagery  in  an  integrated 
automated  system  would  reduce  production  time  by  50  percent  and  product 
completion  time  by  75  percent. 

DPS  Development.  In  1982,  DMA  established  the  Special  Program  Office  for 
Exploitation  Modernization  (disestablished  and  replaced  by  the  DMA  Systems 
Center  in  1987)  to  develop  the  MARK  85  and  MARK  90  systems.  DMA  used  a 
non-DoD  agency  to  contract  for  the  MARK  85  and  MARK  90  systems. 

MARK  85-Initial  Capability  to  Use  Digital  Data.  MARK  85  was  a 
transitional  program  to  upgrade  DMA's  existing  film-based  production  system 
by  providing  an  initial  digital  exploitation  capability  using  new  source  materials. 
MARK  85  was  used  during  Operations  Desert  Shield  and  Desert  Storm  to 
produce  MC&G  products.  During  the  audit,  DMA  was  still  using  MARK  85 
for  the  production  of  MC&G  products.  However,  MARK  85  is  not  a  fully 
integrated  system  and  does  not  have  all  capabilities  intended  for  the  fully 
developed  DPS. 

MARK  90-Integrated  System  Using  Digital  Data.  DMA  developed 
the  MARK  90  to  provide  a  fully  integrated  system  with  an  all-digital  production 
capability  for  designated  products.  DMA  considers  MARK  90,  augmented  by 
certain  MARK  85  capabilities,  to  be  the  DPS.  DMA  cartographers  use 
specialized  workstations  to  perform  tasks,  such  as  identifying  the  imagery's 
specific  location  on  the  earth  and  extracting  digital  data  from  the  imagery. 
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The  DPS  consists  of  seven  segments; 

0  Data  Services, 
o  Production  Management, 

0  Source  Acquisition, 

0  Source  Preparation, 
o  Hardcopy  Exploitation, 
o  Data  Extraction,  and 
0  Product  Generation. 

DMA  developed  portions  of  Production  Management  and  the  majority  of  Source 
Acquisition  and  Hardcopy  Exploitation  for  the  MARK  85,  then  incorporated 
them  in  the  DPS  as  it  was  developed.  Generally,  work  on  a  product  cannot 
conunence  in  a  particular  segment  until  all  work  in  the  previous  segment  has 
been  completed.  The  functions  of  the  seven  DPS  segments  are  described  in 
Appendix  A. 

Contracting,  Size,  and  Cost  of  the  DPS.  The  use  of  digital  data  to 
make  MC&G  products  was  a  new  technology  in  1982.  Because  various 
technical  skills  were  required,  DMA  awarded  separate  contracts  for  each 
segment  of  the  DPS.  Consequently,  the  DPS  has  no  overall  contractor.  DMA, 
with  contractor  engineering  support,  serves  as  the  system  integrator. 
Six  contractors  developed  the  software  in  eight  programming  languages  for  the 
seven  DPS  segments.  DPS  is  used  on  more  than  1,000  workstations  and  is  run 
by  about  8  million  lines  of  contractor-developed  software  code.  DMA  awarded 
cost-plus-award-fee  contracts  to  the  development  contractors  and  expected  the 
DPS  to  be  fully  implemented  in  the  mid-1990's.  The  total  cost  to  develop  the 
DPS,  from  the  inception  of  MARK  85  to  DMA's  determination  of  full 
operational  capability  in  November  1992,  was  $2.6  billion.  In  addition,  DMA 
expended  $322  million  from  FY  1989  through  FY  1993  to  operate  and  maintain 
DPS  hardware  and  software  and  $113  million  to  develop  DPS  modifications  and 
enhancements.  DMA  plans  to  spend  $1.0  billion  from  FY  1994  through 
FY  1999  to  operate  and  maintain  the  DPS  and  to  develop  products  and 
capabilities  that  may  directly  or  indirectly  affect  the  DPS. 

DPS  Products.  The  DPS  is  designed  to  produce  specific  MC&G  products 
categorized  as  aeronautical,  hydrographic,  topographic,  and  digital.  DMA 
changed  the  specific  products  in  each  category  as  the  DPS  was  being  developed 
and  as  customer  requirements  for  MC&G  products  changed.  When  DMA 
selected  development  contractors  for  the  DPS  in  1984,  DMA  planned  that  the 
DPS  would  produce  28  products.  DMA  revised  the  list  in  1989  to  24  products, 
of  which  4  are  no  longer  required.  A  list  of  the  24  products  is  in  Appendix  B. 
The  changes  made  to  the  DPS  products  since  1984  are  detailed  in  Appendix  C. 
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DoD  Hotline  Allegations.  Two  anonymous  DoD  Hotline  complaints, 
submitted  during  1992,  alleged  improprieties  in  the  management  of  DMA  and  in 
the  development  and  operation  of  DMA  projects.  One  Hotline  complaint 
alleged  that  the  overall  management  of  DMA  was  ineffective  and  that  guidelines 
were  bypassed  for  funding  and  contracting  of  DMA  projects.  The  other  Hotline 
complaint  alleged  improprieties  in  the  funding,  contracting,  development,  and 
operation  of  the  DPS. 


Objective 


The  audit  objective  was  to  evaluate  the  validity  of  the  Hotline  allegations  that 
sound  management  practices  were  not  being  used  in  the  acquisition  and 
operation  of  DMA  projects.  Specifically,  we  evaluated  the  policies  and 
procedures  used  to  manage  the  development  and  acquisition  of  the  DPS. 


Scope  and  Methodology 


Audit  Work  Performed.  We  made  the  audit  in  conjunction  with  auditors  from 
another  Government  agency  and  reviewed  20  of  the  25  allegations.  DoD 
Hotline  officials  referred  four  allegations  either  to  Departmental  Inquiries, 
Office  of  the  Inspector  General,  DoD,  or  Inspector  General,  DMA,  for 
evaluation.  We  referred  one  allegation  to  the  Defense  Criminal  Investigative 
Service,  Office  of  the  Inspector  General,  DoD,  for  evaluation.  A  summary  of 
the  20  allegations  and  audit  results  is  in  Appendix  D. 

We  reviewed  DMA's  overall  strategy  and  its  policies  for  project  development 
and  examined  DPS  program  documentation,  dated  from  December  1982  to 
March  1994,  that  related  to  the  development  and  operation  of  the  DPS. 
Specifically,  we  reviewed  DPS  plans  and  system  specifications,  contracts, 
budgets,  test  plans  and  results,  training  plans,  configuration  management  plans, 
production  readiness  reviews,  production  plans,  and  status  reports  on  the 
development  of  the  system.  In  addition,  we: 

o  identified  and  analyzed  software  discrepancy  reports,  dated  from 
FY  1988  to  FY  1993,  by  DPS  segment; 

o  reconciled  the  funds  allotted  to  DMA  with  obligation  documents; 

o  selected  (judgmentally)  project  tasks  that  were  to  be  completed  from 
April  through  October  1992  and  determined  how  tasks  were  contracted  and 
funds  were  used; 

o  analyzed  the  number  of  personnel  trained  on  the  DPS  and  the  number 
of  workstations  used  on  the  DPS; 
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o  interviewed  DMA  officials  and  contractor  personnel  who  either  were 
involved  in  the  DPS  development  or  had  knowledge  of  its  production  status;  and 

o  interviewed  DMA  officials  and  cartographers  who  used  the  system 
and  Office  of  Secretary  of  Defense  officials  having  knowledge  of  DoD  major 
acquisition  procedures  and  related  oversight  responsibilities. 

Technical  Assistance  and  Computer-Processed  Data.  A  software  engineer 
from  the  technical  assessment  staff  of  the  Office  of  the  Inspector  General,  DoD, 
assisted  the  Inspector  General,  DoD,  auditors  in  reviewing  procedures  for 
controlling  changes  to  system  documentation.  Also,  the  auditors  used 
computer-processed  data  from  DMA's  Automated  Configuration  Management 
System  to  identify  the  number  of  software  discrepancy  reports.  The  auditors 
determined  the  accuracy  of  the  data  by  reviewing  the  controls  on  data  entry  for 
the  Automated  Configuration  Management  System.  Organizations  visited 
during  the  audit  are  listed  in  Appendix  H. 

Auditing  Standards.  This  program  audit  was  made  from  March  1993  through 
March  1994  in  accordance  with  auditing  standards  issued  by  the  Comptroller 
General  of  the  United  States,  as  implemented  by  the  Inspector  General,  DoD. 
Accordingly,  the  audit  included  such  tests  of  internal  controls  as  were 
considered  necessary. 


Internal  Controls 


Adequacy  of  Internal  Controls.  We  did  not  evaluate  DMA's  implementation 
of  the  DoD  Internal  Management  Control  Program  as  defined  by  DoD 
Directive  5010.38,  "Internal  Management  Control  Program,"  April  14,  1987. 
We  limited  our  evaluation  of  internal  controls  to  funding,  contracting, 
development,  and  operation  of  the  DPS.  The  audit  identified  material  internal 
control  weaknesses  in  identifying  all  customer  requirements,  analyzing  software 
problems,  maintaining  configuration  management  of  the  DPS,  and  defining 
modifications  needed  for  the  DPS.  DMA  did  not  report  those  weaknesses  in  its 
Internal  Management  Control  Program  report  because  they  did  not  believe  the 
weaknesses  were  material.  Implementation  of  all  the  recommendations  in 
Finding  A  will  correct  the  internal  control  weaknesses.  A  copy  of  the  report 
will  be  provided  to  the  senior  official  responsible  for  internal  controls  within 
DMA. 


Prior  Audits  and  Other  Reviews 


Office  of  the  Inspector  General,  DoD,  Audit  Report  No.  94-076, 
"Contracting  Authority  at  the  Defense  Mapping  Agency,"  April  1,  1994. 
The  report  discusses  the  propriety  of  contracting  support  provided  to  the  DPS 
by  a  non-DoD  agency  and  the  benefits  that  could  be  attained  if  DMA  performed 
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the  contracting  for  the  DPS.  The  report  recommended  that  the  DMA  develop  a 
plan  to  transfer  contracting  responsibility  to  the  DMA.  DMA  concurred  in  the 
recommendation  and  began  developing  a  plan  to  transition  DPS  contracting  to 
the  DMA.  The  report  is  classified. 

Office  of  the  Inspector  General,  DoD,  Inspection  Report  No.  89-INS-02, 
"Report  on  the  Inspection  of  the  Defense  Mapping  Agency,"  February  7, 
1989.  The  report  discusses  the  results  of  an  overall  inspection  of  DMA 
performed  in  FY  1988.  The  report  recommended  that  the  Under  Secretary  of 
Defense  for  Acquisition  (now  the  Under  Secretary  of  Defense  for  Acquisition 
and  Technology)  conduct  a  Milestone  n  review  of  the  program  management 
process  for  the  DPS.  The  Under  Secretary  responded  that  the  Defense 
Acquisition  Board  process  was  inappropriate  for  reviewing  the  management  and 
documentation  of  the  DPS.  Our  current  audit  showed  that  the  Defense 
Acquisition  Board  should  review  the  DPS  (Finding  B).  Report  No.  89-INS-02 
also  recommended  that  the  Assistant  Secretary  of  Defense  (Command,  Control, 
Communications  and  Intelligence)  conduct  the  equivalent  of  a  Milestone  n 
review  of  the  DPS.  The  Assistant  Secretary  initially  concurred,  but  did  not 
conduct  the  review.  Instead,  personnel  on  his  staff  attended  the  DPS  critical 
design  review  in  December  1988  and  DMA  briefings  on  the  DPS.  Our  current 
audit  determined  that  lack  of  oversight  by  the  Assistant  Secretary  contributed  to 
problems  in  the  DPS  development  (Finding  B).  Report  No.  89-INS-02  further 
recommended  that  DMA  reduce  the  security  classification  of  DPS 
documentation  to  facilitate  the  transfer  of  information  to  DMA  employees.  In 
June  1989,  DMA  reduced  the  security  level  of  some  aspects  of  the  program. 
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Finding  A.  Management  of  Digital 
Production  System 
Development 

The  Defense  Mapping  Agency's  development  of  the  Digital  Production  System 
needed  improvements.  The  Defense  Mapping  Agency  did  not: 

o  identify  all  product  specifications  for  the  Flight  Information 
Publications  and  Automated  Air  Facilities  Information  File  products, 

o  analyze  causes  of  software  problems, 

o  maintain  configuration  management  of  the  Digital  Production 
System,  and 

o  define  modifications  needed  for  the  Digital  Production  System. 

As  a  result,  customer  requirements  for  the  Flight  Information  Publications  and 
Automated  Air  Facilities  Information  File  products  in  the  Digital  Production 
System  were  not  met.  Further,  until  the  Digital  Production  System  is  fully 
operational,  the  Defense  Mapping  Agency  may  experience  further  delays  in  the 
production  of  the  24  scheduled  MC&G  products  and  may  unnecessarily  expend 
resources  for  system  modifications. 


Background 


Program  Management  of  the  DPS  Development.  The  DMA  Systems  Center 
was  primarily  responsible  for  developing  the  DPS.  To  help  manage  the 
development  of  the  DPS,  DMA  established  a  series  of  review  boards.  The 
review  boards  performed  such  functions  as  approving  the  configuration  of  the 
DPS,  determining  the  production  readiness  of  the  DPS,  and  controlling  the 
deployment  of  the  DPS  to  the  production  components.  Senior  officials  from 
DMA  Headquarters,  Systems  Center,  and  the  production  components  generally 
comprised  the  review  boards.  By  including  those  representatives,  DMA  wanted 
to  ensure  that  the  DPS  development  was  well-coordinated  and  represented  the 
system  that  would  meet  DMA's  needs. 


Identifying  Customer  Requirements  for  the  DPS 


Software  Requirements  for  the  Flight  Information  Publications  and 
Automated  Air  Facilities  Information  File  Products.  The  Systems  Center  did 
not  develop  comprehensive  product  specifications  for  the  Flight  Information 
Publications  and  Automated  Air  Facilities  Information  File  products.  In  March 
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1993,  Systems  Center  testing  determined  that  the  DPS  software  for  the  Flight 
Information  Publications  and  the  Automated  Air  Facilities  Information  File 
products  did  not  work  properly.  The  Systems  Center  assembled  a  team  of 
DMA  and  contractor  personnel  on  March  9,  1993,  to  review  the  software 
development  status  for  the  two  products.  On  March  18,  1993,  the  team  issued  a 
report  identifying  nine  problems  in  the  development  of  software  for  the  Flight 
Information  Publications  and  Automated  Air  Facilities  Information  File 
products.  For  example,  the  team  determined  that  the  Systems  Center  did  not 
draft  the  product  specifications  according  to  standards  and  did  not  formally 
review  the  specifications.  In  addition,  the  Systems  Center  had  not  incorporated 
18  customer-requested  changes  in  specifications  for  the  Flight  Information 
Publications  and  the  Automated  Air  Facilities  Information  File  products.  To 
correct  those  problems,  the  contractor  performed  extensive  work  during  1993 
on  the  software  for  the  products.  The  revised  software  did  not  pass  all 
additional  factory  acceptance  tests  until  December  1993,  more  than  1  year  after 
DMA  determined  on  Novembers,  1992,  that  the  DPS  had  reached  full 
operational  capability. 

Additional  Software  Changes.  At  the  time  of  the  audit,  DMA  customers  were 
identifying  changes  to  their  requirements  for  the  Flight  Information 
Publications.  Because  Flight  Information  Publications  are  critical  to  flight 
safety,  the  Systems  Center  must  change  the  product  specification  and  develop  or 
modify  software  to  meet  changes  in  customer  requirements.  The  Systems 
Center  estimated  that  the  DPS  will  not  be  capable  of  meeting  current  customer 
specifications  for  Flight  Information  Publications  products  until  at  least  late 
FY  1995. 

Contracted  Maintenance  of  Software.  As  new  information  was  obtained  for 
the  Flight  Information  Publications,  DMA  used  contractors  to  change  the  data. 
DMA  budgeted  about  $2.0  million  and  $1.8  million  for  FYs  1994  and  1995, 
respectively,  for  the  data  maintenance.  DMA  planned  to  use  the  DPS  to  change 
data  for  the  Flight  Information  Publications.  Because  of  the  number  of 
individual  products  within  the  Flight  Information  Publications,  DMA  planned  to 
phase  out  the  contractor  maintenance  through  FY  1997.  Until  the  DPS  is 
capable  of  updating  needed  data  for  individual  products,  the  DMA  must 
continue  to  contract  for  die  maintenance  of  the  Flight  Information  Publications 
products.  To  prevent  further  delays  in  the  development  of  the  software  for  the 
Flight  Information  Publications  products  and  to  avoid  paying  data  maintenance 
charges  for  maintenance  that  should  be  done  by  the  DPS,  the  Systems  Center 
must  ensure  product  specifications  are  complete. 


Analyzing  Problems  in  DPS  Software  Development 


Reporting  Procedures.  DMA  persoimel  recorded  problems  in  the  DPS  in 
discrepancy  reports.  A  discrepancy  report  contains  information,  such  as  a 
description  of  the  problem,  the  date  it  occurred,  its  priority  and  status,  the 
organization  responsible  for  fixing  the  discrepancy,  and  the  reason  why  the 
problem  occurred.  DMA  personnel  submitted  the  discrepancy  reports  into 
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DMA's  Automated  Configuration  Management  System.  The  Automated 
Configuration  Management  System  generates  data  for  engineering  review  boards 
responsible  for  reviewing  the  data,  determining  causes  for  the  problems,  and 
prioritizing  their  correction. 

Reasons  for  Problems.  The  Systems  Center  developed  category  codes  to 
represent  causes  of  problems  identified  in  discrepancy  reports.  However,  the 
engineers  responsible  for  determining  the  causes  of  the  problems  did  not  enter 
all  category  codes  in  the  final  discrepancy  reports  or  in  the  Automated 
Configuration  Management  System,  as  shown  in  the  table  below.  Procedures 
for  using  the  Automated  Configuration  Management  System  do  not  require  the 
category  code  field  to  be  completed.  Without  all  category  codes,  engineers 
could  not  track  development  problems. 

Category  Codes  Omitted  from  Closed  Discrepancy  Reports 
(August  1  to  September  30,  1993) 


Discrepancy  Reports 


DPS  Segment 
Data  Services 
Production  Management 
Source  Acquisition 
Source  Preparation 
Hard  Copy  Exploitation 
Data  Extraction 
Product  Generation 


Total 

Without  a 
Category  Code 

444 

21 

694 

467 

82 

46 

665 

18 

7 

4 

1,297 

45 

784 

168 

Some  problems  identified  in  the  discrepancy  reports  could  have  been  prevented 
if  the  engineering  review  boards  had  been  able  to  develop  a  methodology  for 
analyzing  overall  causes  and  their  trends.  By  identifying  systemic  problems, 
DMA  may  be  able  to  make  changes  to  software  requirements  before  production 
is  affected  and  will  be  able  to  utilize  budget  resources  more  efficiently.  As 
discussed  below,  DMA  expects  to  encounter  many  more  software  defects  that 
will  affect  production  and  satisfaction  of  customer  requirements. 

Software  Defects  in  the  DPS.  The  system  integration  contractor  prepared  the 
"Software  Maintenance  Study  for  DMA,"  August  1,  1993,  to  estimate  the 
number  of  defects  in  the  DPS  and  to  evaluate  maintenance  requirements. 
Studies  made  by  the  software  industry  of  completed  software  systems  have 
determined  that  in  any  major  software  development,  a  number  of  defects  exist  in 
the  software  delivered  to  the  user.  Inherent  defects  stem  mainly  from  the 
difficulty  in  adequately  defining  the  users'  requirements  and  in  writing  software 
code  that  satisfies  user  requirements.  To  estimate  the  number  of  defects  in  the 
DPS  software,  the  system  integration  contractor  used  a  computer  model  based 
on  the  experiences  of  other  large-scale  DoD  development  programs.  The 
system  integration  contractor  estimated  that  the  DPS  had  about  82,000  defects 
when  the  software  was  delivered  to  DMA  at  the  end  of  the  factory  acceptance 
tests. 
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DPS  Compared  to  Similar  Programs.  The  system  integration  contractor 
estimated  that  for  other  comparable  DoD  programs,  program  officials 
discovered  about  70  percent  of  the  inherent  defects  by  production  start-up.  As 
of  November  5,  1992,  when  DMA  determined  that  the  DPS  had  reached  full 
operational  capability,  DMA  had  discovered  only  15,265  (19  percent)  of  the 
expected  82,000  defects.  As  of  August  31,  1993,  DMA  had  discovered  only 
24,720  (30  percent)  of  the  expected  82,000  defects.  It  is  realistic  to  expect  that 
DMA  will  discover  about  81  percent  of  the  estimated  defects  while  the  DPS  is 
being  used  for  production  operations.  By  using  the  DPS  before  it  is  fully  tested 
and  before  most  errors  are  identified,  DMA  gains  greater  familiarity  of  the 
system,  tests  the  system  in  an  operational  environment,  and  gets  some 
production  work  accomplished.  However,  defects  discovered  after  the  software 
has  been  delivered  are  harder  and  more  expensive  to  fix  than  defects  discovered 
during  initial  development  and  cause  delays  in  production. 


Configuration  Management  for  the  DPS 


Purpose  of  Configuration  Management.  As  of  March  15,  1994,  DMA  had 
not  resolved  deficiencies  in  configuration  management  identified  by  a  1993 
Systems  Center  review.  Configuration  management  is  an  integral  part  of  the 
systems  engineering  management  process.  The  role  of  configuration 
management  is  to  identify  the  functional  and  physical  characteristics  of  system 
components  during  the  system's  life  cycle.  Configuration  management  is 
particularly  important  for  die  DPS  since  it  was  developed  by  several  contractors 
and  since  numerous  changes  are  being  made  to  the  DPS  to  fix  software  defects 
and  to  add  capabilities.  Changes  to  a  system  are  controlled  and  documented 
using  configuration  management  procedures  to  ensure  all  users  and  developers 
are  aware  of  the  current  configuration  status  of  the  system.  Weak  configuration 
management  increases  the  risk  that  software  may  not  be  adequately  documented. 
Without  adequate  documentation,  incorrect  software  modifications  could  occur 
and  resomces  may  be  wasted. 

Systems  Center  Review  of  Configuration  Management.  During  August 
through  November  1992,  Systems  Center  personnel,  in  conjunction  with  the 
three  production  components  and  the  system  integration  contractor,  reviewed 
the  configuration  management  process  and  organization  used  by  the  DPS 
contractors  and  DMA.  In  addition,  the  personnel  reviewed  the  implementation 
of  configuration  management  procedures  for  documentation,  software, 
hardware,  and  discrepancy  reporting.  The  review  identified  15  deficiencies  in 
DMA's  configuration  management. 

During  April  through  June  1993,  Systems  Center  personnel,  again  in 
conjunction  with  personnel  from  the  three  production  components  and  the 
system  integration  contractor,  conducted  a  more  detailed  configuration 
management  audit  and  followed  up  on  the  deficiencies  identified  in  the 
1992  review.  The  Systems  Center  review  determined  that  DMA  had  corrected 
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only  4  of  the  15  identified  deficiencies.  The  1993  Systems  Center  review 
concluded  that  configuration  control  of  the  DPS  could  be  disrupted  if  actions 
were  not  taken  to  fix  the  deficiencies. 

Establishment  of  a  Configmation  Management  Steering  Group.  To 
identify  solutions  to  DMA's  confi^ation  management  deficiencies  and  to 
monitor  the  corrective  actions,  the  Director,  Systems  Center,  formed  a  steering 
group  in  September  1993.  The  steering  group  consisted  of  personnel  from 
Headquarters,  DMA,  the  Systems  Center,  die  production  components,  and  the 
system  integration  contractor. 

Correction  of  Configuration  Management  Deficiencies.  To  determine 
the  status  of  actions  t^en  to  resolve  the  11  deficiencies,  we  reviewed  the 
minutes  of  the  steering  group's  meetings  from  October  1993  through  February 
1994,  and  interviewed  the  Chairman  of  the  Steering  Group.  The  minutes 
showed  that  the  Steering  Group  considered  one  deficiency  resolved.  The 
Steering  Group  also  had  established  formal  target  dates  to  resolve  2  of  the 
remaining  10  deficiencies.  One  of  those  two  deficiencies  was  resolved  when 
DMA  reorganized  the  Systems  Center  on  March  20,  1994,  and  administratively 
established  a  configuration  management  organization  within  the  Systems  Center, 
although  it  had  not  yet  staffed  that  organization. 

As  of  February  3,  1994,  the  Systems  Center  was  still  pursuing  solutions  for  the 
remaining  eight  deficiencies.  Those  remaining  deficiencies  included 
configuration  management  training,  configmation  control  records,  and 
configuration  control  guidelines  for  the  engineering  review  boards.  DMA  needs 
to  follow  through  on  physically  establishing  and  staffing  an  effective 
organization  for  configuration  management  and  on  developing  solutions  to 
remaining  problems.  Without  effective  configuration  management,  resources 
could  be  wasted  through  changes  that  do  not  correct  system  software  defects  and 
through  inappropriate  system  modifications. 


Program  Management  Plan  for  DPS  Modifications 


Short-  and  Long-Term  DPS  Modifications.  The  DMA  had  not  developed  a 
program  management  plan  that  clearly  defined  the  objectives  and  development 
strategy  for  the  short-  and  long-term  modifications  planned  for  the  DPS.  A 
program  management  plan  describes  the  program's  history  and  objectives  and 
serves  as  the  vehicle  for  guiding  the  overall  program.  Supporting  plans  for  all 
phases  of  system  development  include  plans  for  the  test  and  evaluation  of  the 
system  <l»ring  development  and  maintenance  for  the  developed  system.  As 
system  development  progresses,  program  plans  are  revised  to  include  more 
details. 

From  FY  1994  through  FY  1999,  DMA  planned  to  use  about  $185  million  in 
Research,  Development,  Test  and  Evaluation  funds  either  for  modifying  the 
DPS  or  for  developing  new  products.  In  addition,  the  DMA  planned  to  spend 
$220  million  from  FY  1994  through  FY  1999  on  research  and  development 


12 


Finding  A.  Management  of  Digital  Production  System  Development 


efforts  that  may  directly  or  indirectly  result  in  modifications  to  DPS  operations 
or  products.  Since  Ae  Systems  Center  is  already  modifying  the  DPS,  a 
program  management  plan  would  help  to  verify  that  modifications  to  DPS  are 
completed  according  to  priority  and  that  resources  are  not  wasted. 

Modifications  of  DPS  Products  and  Operations.  Expected  modifications  to 
the  DPS  include  new  products,  equipment,  and  procedures. 

0  Customers  require  new  products,  such  as  a  1:100,000  Topographic 
Line  Map  and  a  Tactical  Terrain  Data  digital  product. 

0  DPS  equipment  will  have  to  be  replaced  because  of  changes  in 
technology  since  DPS  development  started.  DMA  has  already  made  some 
equipment  changes.  For  example,  a  contractor-developed  switch  in  the  Data 
Services  segment  was  replaced  with  a  commercial  off-the-shelf  interface.  The 
interface  is  more  reliable  and  is  more  easily  maintained  because  of  its 
commercial  applications. 

o  New  DPS  operational  procedures  are  needed  because  of  changes  in 
the  world  order.  The  DPS  was  designed  for  a  long-term  production  program, 
concentrating  primarily  on  the  known  adversa^,  the  Soviet  Union,  ^e  DPS 
must  be  modified  to  provide  an  efficient  crisis-reaction  capability  to  better 
handle  various  operations,  such  as  Operation  Restore  Hope  in  Somalia. 

A  maintenance  advisory  panel  of  personnel  from  industry  and  Government 
convened  by  the  Director,  DMA,  in  1993  also  identified  needed  changes  in  DPS 
development  and  modification  procedures.  In  the  report,  "Advisory  Panel 
Report  on  Digital  Production  System  Maintenance  Management, "  July  22, 
1993,  the  advisory  panel  made  10  recommendations  regarding  the  further 
development  and  maintenance  of  the  DPS.  Those  recommendations  included 
establishing  a  separate,  permanent  development  and  test  capability  that 
replicates  the  DPS  and  replacing  the  development  contractors  with  a  single 
contractor  for  maintaining  all  DPS  segments.  The  advisory  panel  also 
recommended  that  DMA  prepare  a  strategic  plan  for  DPS  modifications  that 
identified  cost,  schedule,  and  performance  planning.  The  Director,  DMA, 
directed  that  each  of  the  advisory  panel's  10  recormnendations  be  implemented. 
Although  the  Systems  Center  developed  the  DPS  using  a  non-DoD  development 
process,  the  Director,  Systems  Center,  intends  to  follow  DoD  Directive  5000.1, 
"Defense  Acquisitions,"  February  23,  1991,  for  future  modifications. 

Additional  details  on  the  development  strategy  are  in  Appendix  E.  DMA  must 
develop  a  program  management  plan  for  all  modifications  to  DPS  development 
to  ensure  that  resources  are  used  wisely. 
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Recommendations,  Management  Comments,  and  Audit 
Response 


We  recommend  that  the  Director,  Defense  Mapping  Agent^: 

1.  Implement  procedures  for  identifying  and  coordinating  customer 
technical  requirements  for  the  Flight  Information  Publications  and 
Automated  Air  Facilities  Information  File  products  before  specifications  are 
given  to  the  contractor  for  development. 

2.  Revise  procedures  for  discrepancy  reporting  for  the  Digital 
Production  System  to  require  that  the  category  codes  identifying  causes  of 
problems  be  entered  in  discrepancy  reports  and  in  the  Defense  Mapping 
Agency's  Automated  ConHguration  Management  System. 

3.  Establish  procedures  and  a  methodology  for  analyzing  causes  of 
and  trends  in  problems  identified  in  discrepancy  reports  on  the  Digital 
Production  System. 

4.  Assign  personnel  to  the  configuration  management  organization 
responsible  for  developing  conHguration  management  procedures  and  for 
providing  oversight  of  the  conflguration  management  function  in  the 
Defense  Mapping  Agency. 

5.  Develop  and  implement  plans  mth  completion  dates  for  the 
correction  of  the  deficiencies  identified  in  the  Systems  Center  1993 
configuration  management  review. 

6.  Develop  a  comprehensive  program  management  plan  that  defines 
the  modiDcations  to  be  made  to  the  Distal  Production  System  and  that 
details  how  and  when  the  modifications  be  accomplished. 

Management  Comments  on  the  Recommendations  and  Internal  Controls. 
The  Assistant  Secretary  of  Defense  (Command,  Control,  Communications  and 
Intelligence)  submitted  consolidated  comments  for  himself;  the  Under  Secretary 
of  Defense  for  Acquisition  and  Technology;  and  the  Director,  DMA.  The 
Assistant  Secretary  concurred  in  all  recommendations,  but  did  not  agree  that 
material  weaknesses  existed  in  DMA's  process  for  customer  requirement 
identification,  software  problem  analysis,  configuration  management,  and  DPS 
modification  definition.  The  Assistant  Secretary  stated  that  DMA's  internal 
control  procedures  had  identified  each  of  the  weaknesses  and  that  DMA  had 
either  implemented  or  was  in  the  process  of  implementing  solutions. 

Audit  Response.  Management  comments  were  responsive  to  the 
recommendations.  We  considered  the  internal  control  weaknesses  material 
because  DMA  experienced  delays  in  the  development  of  the  DPS  and  resources 
could  have  been  wasted.  However,  since  management  concurred  in  the 
recommendations  the  apparent  disagreement  could  be  because  of  semantics. 
The  corrective  actions  t^en  or  planned  to  be  taken  will  resolve  the  internal 
control  weaknesses  and  no  fiirther  management  comments  are  required. 
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Production  System 
Development 

The  development  and  modification  of  the  Digital  Production  System 
lacked  sufficient  oversight.  Because  the  Digital  Production  System  was 
not  subject  to  acquisition  policies  in  DoD  directives  and  instructions,  the 
Defense  Acquisition  Board  had  not  reviewed  the  development  of  the 
Digital  Production  System.  With  DMA's  decision  in  1994  to  transfer 
contracting  responsibilities  for  the  Digital  Production  System  from  a 
non-DoD  agency  to  the  DMA,  the  Digital  Production  System  is  subject 
to  DoD  acquisition  policies.  Further,  the  Assistant  Secretary  of  Defense 
(Command,  Control,  Communications  and  Intelligence)  relied  on  a 
Defense  Mapping  Agency  liaison  officer  to  provide  oversight  of  the 
Digital  Production  System.  If  sufficient  oversight  is  not  provided,  the 
Defense  Mapping  Agency  may  expend  resources  for  unneeded 
modifications  and  experience  further  delays  in  the  production  of  MC&G 
products. 


Background 


Categories  of  Software  Systenns.  DoD  categorizes  software  systems  as  either 
mission-critical  computer  resources  or  automated  information  systems  (AISs), 
depending  on  how  the  software  is  used.  A  mission-critical  computer  resource  is 
physically  part  of,  dedicated  to,  or  essential  to  a  weapon  system.  An  AIS 
performs  functions,  such  as  collecting,  processing,  transmitting,  and  displaying 
information,  and  is  not  part  of  a  weapon  system.  Mission-critical  computer 
resources  and  AISs  are  expected  to  be  constantly  changing  and  evolving  as  new 
requirements  are  identified  and  implemented.  Ifie  DPS  is  an  AIS. 

Software  Development  Policies  and  Procedures.  DoD  Directive  8120.1, 
"Life-Cycle  Management  (LCM)  of  Automated  Information  Systems  (AISs)," 
January  14,  1993,  and  DoD  Directive  8120.2,  "Automated  Information  System 
(AIS  Life-Cycle  Process,  Review,  and  Milestone  Approval  Procedures)," 
January  14,  1993,  establish  policies  and  procedures  for  the  development  of 
AISs.  The  Major  Automated  Information  System  Review  Council  provides 
oversight  of  the  development  of  AISs.  The  Assistant  Secretary  of  Defense 
(Command,  Control,  Communications  and  Intelligence)  is  responsible  for  the 
Major  Automated  Information  System  Review  Council.  DoD  Directive  5000.1, 
"Defense  Acquisitions,"  February  23,  1991,  and  DoD  Instruction  5000.2, 
"Defense  Acquisition  Management  Policies  and  Procedures,"  February  23, 
1991,  establish  policies  and  procedures  for  the  acquisition  of  major  and 
nonmajor  weapon  systems.  Although  most  of  the  terms  of  DoD 
Directive  5000.1  do  not  apply  to  AISs,  DoD  Directive  8120.1  requires  that 
AISs  be  managed  as  consistently  as  possible  according  to  policies  in  DoD 
Directive  5000.1.  In  addition,  DoD  Instruction  8120.2  states  that  "AISs  that 


15 


Finding  B.  Oversight  of  Digital  Production  System  Development 


exceed  the  Defense  Acquisition  Board  thresholds  shall  be  forwarded  to  the 
Defense  Acquisition  Board  for  review."  According  to  DoD  Instruction  5000.2, 
"Defense  Acquisition  Management  Policies  and  Procedures,"  February  23, 
1991,  the  threshold  for  Defense  Acquisition  Board  review  of  an  AIS  is  a  total 
expenditure  for  research  and  development  of  more  than  $300  million  in 
FY  1990  constant  dollars  (acquisition  category  I). 


Oversight  of  the  DPS 


External  Review  of  the  DPS.  Because  DoD  considered  DPS  a  classified 
program  and  because  DMA  used  non-DoD  channels  and  practices  in  the  DPS 
development,  the  Assistant  Secretary  of  Defense  (Command,  Control, 
Communications  and  Intelligence)  did  not  require  the  DMA  to  comply  with 
DoD  acquisition  policies  that  unclassified  programs  must  follow.  As  a  result, 
the  Major  Automated  Information  System  Review  Council  exempted  the  DPS 
from  review.  Inspector  General,  DoD,  Inspection  Report  No.  89-INS-02, 
"Report  on  the  Inspection  of  the  Defense  Mapping  Agency, "  February  7,  1989, 
recommended  that  the  Under  Secretary  of  Defense  for  Acquisition  (now  the 
Under  Secretary  of  Defense  for  Acquisition  and  Technology)  conduct  a 
Milestone  II  review  of  the  DPS.  In  response,  die  Under  Secretary  declined  to 
review  the  DPS,  because  he  deemed  the  Defense  Acquisition  Board  process 
inappropriate  for  reviewing  the  DPS  development. 

To  provide  some  external  oversight,  the  former  Director,  DMA,  periodically 
informed  the  Assistant  Secretary  of  Defense  (Command,  Control, 
Communications  and  Intelligence)  on  the  status  of  overall  DPS  development. 
As  part  of  the  annual  budget  process,  DMA  Comptroller  persoimel  provided 
briefings  on  DPS  development  to  congressional  committee  staff  with  a 
demonstrated  need  to  know.  In  addition,  DMA  convened  independent  panels  in 
FY  1988  and  FY  1993  to  review  the  DPS.  The  Red  Team,  composed  of 
personnel  from  industry,  academia,  and  Government,  reviewed  the  overall 
development  of  the  program  in  FY  1988.  As  previously  discussed,  the 
Maintenance  Advisory  Panel  reviewed  maintenance  requirements  for  the  DPS  in 
FY  1993.  Those  reviews  helped  the  DMA  focus  on  key  development  and 
maintenance  issues. 

Review  of  Programs  at  Major  Milestones.  For  programs  that  are  not 
exempted,  the  Major  Automated  Information  System  Review  Council  or 
Defense  Acquisition  Board  reviews  costly  programs,  such  as  the  DPS,  as  the 
program  transitions  to  each  major  milestone  in  the  development  process.  The 
milestone  reviews  determine  whether  the  program  has  made  adequate  progress 
and  whether  appropriate  management  controls  are  in  place  for  the  program  to 
proceed  to  the  next  milestone.  Programs  cannot  continue  unless  criteria 
established  for  moving  to  the  next  development  phase  have  been  met. 

Status  of  DPS  Development  and  Production.  Significant  progress  had  been 
made  in  the  development  of  the  DPS,  but  the  capability  to  produce  all  products 
had  not  been  demonstrated  and  production  was  limited.  As  previously 
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discussed,  the  Systems  Center  estimated  that  the  DPS  will  not  be  capable  of 
meeting  all  customer  requirements  for  Flight  Information  Publications  products 
until  late  FY  1995.  In  addition,  because  customer  requirements  had  changed, 
additional  work  was  needed  to  make  the  Digital  Feature  Analysis  Data, 
Level  II,  product  releasable  to  the  customers.  Systems  Center  integration 
officials  estimated  that  the  software  for  the  revised  Digital  Feature  Analysis 
Data,  Level  II,  product  would  be  implemented  in  the  DPS  in  mid-1994. 
Further,  as  of  September  9,  1993,  DMA  had  not  demonstrated  whether  the  DPS 
could  produce  seven  other  products  by  either  producing  a  finished  version  or 
testing^  the  finishing  of  the  seven  products  listed  below. 

0  Air  Target  Chart 

o  Joint  Operations  Graphic  (Radar) 

o  Tactical  Pilotage  Chart 

o  Operational  Navigation  Chart 

0  Point  Positioning  Data  Base 

o  Harbor  and  Approach  Chart,  and 

o  Combat  Chart 

DMA  production  and  integration  officials  told  us  that  the  capability  to  produce 
those  seven  products  had  been  demonstrated  by  the  test  or  production  of  similar 
products  and  that  limited  time  was  available  for  testing.  Until  all  products  are 
fully  produced  or  tested,  DMA  cannot  be  assured  of  DPS  capabilities. 

To  fully  demonstrate  the  capabilities  of  the  DPS,  more  products  need  to  be 
finished.  The  number  of  finished  products  was  limited  because  the  majority  of 
the  work  done  on  the  DPS  was  in  preparation  for  finishing  products.  Work 
began  in  July  1992  in  the  Source  Preparation  segment  on  determining  the 
position  of  imagery  on  the  earth.  Work  began  in  November  1992  in  the  Data 
Extraction  segment  to  extract  information  from  the  imagery  and  to  enter  it  in  the 
MC&G  data  base.  However,  as  of  January  31,  1994,  only 

one  1:50,000^  Topographic  Line  Map  and  one  Digital  Terrain  Elevation  Data 
product  had  been  finished  through  the  Product  Generation  segment.  The  DPS 
produced  Point  Positioning  Program  products,  but  those  products  were 
produced  in  the  Source  Preparation  segment,  not  in  the  Data  Extraction  and 
Product  Generation  segments. 


^In  addition  to  the  seven  products  discussed,  DMA  was  not  testing  four  other 
products  that  were  no  longer  required;  they  will  be  replaced  in  FY  1996  by  a 
new  single  product. 

^1:50, 000  is  the  scale  of  the  map.  For  a  1:50,000  Topographic  Line  Map,  the 
actual  geographical  area  shown  on  the  map  is  approximately  50,000  times  larger 
than  depicted  on  the  map. 
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Production  operations  are  key  to  demonstrating  the  effectiveness  of  the  DPS. 
The  Systems  Center  transferred  control  over  DPS  production  operations  to  the 
DMA  production  components  on  October  1,  1993.  For  DMA's  FY  1994 
production  program,  tihe  DMA  production  components  committed  to  finishing 
products  through  the  Product  Generation  segment.  The  number  of  work  years 
planned  as  of  February  18,  1994,  for  the  DPS  FY  1994  production  program  and 
the  number  of  products  to  be  produced  are  shown  in  Appendix  F. 

Oversight  Needed  hy  the  Defense  Acquisition  Board.  Effective  oversight  is 
critical  to  ensure  the  best  use  of  allocated  resources.  The  DPS  is  an  AIS  and 
would  have  been  reviewed  by  the  Major  Automated  Information  System  Review 
Council,  if  the  DPS  had  not  been  exempted  from  review.  Due  to  total 
development  costs  and  planned  expenditures,  the  Defense  Acquisition  Board 
should  review  the  DPS.  The  DMA  spent  $2  billion  in  Research,  Development, 
Test  and  Evaluation  funds  in  developing  the  DPS.  DMA  also  planned  to  spend 
about  $185  million  more  from  FY  1994  through  FY  1999  in  Research, 
Development,  Test,  and  Evaluation  funds  either  for  enhancing  the  capabilities  of 
the  DPS  or  for  developing  new  products.  In  addition,  DMA  planned  to  spend 
$220  million  from  FY  1994  through  FY  1999  on  research  and  development 
efforts  that  may  directly  or  indirectly  result  in  modifications  to  DPS  operations 
or  products.  DPS  expenditures  exceed  the  dollar  threshold  for  a  Defense 
Acquisition  Board  review,  and  die  capability  to  produce  all  products  has  not 
been  verified.  Therefore,  the  Board  should  conduct  a  Milestone  IV  (Major 
Modification  Approval)  review  of  the  DPS. 

Production  Backlog.  A  Defense  Acquisition  Board  review  of  DPS 
development  is  also  needed  because  MC&G  products  are  critical  for  military 
operations.  At  the  time  of  the  audit,  DoD  lacked  adequate  MC&G  coverage  for 
many  parts  of  the  world.  As  of  November  1993,  DMA  had  a  production 
backlog  of  about  140,000  customer  requirements.  Of  that  production  backlog, 
customers  considered  about  83,000  requirements  to  be  critical  or  essential  to  Ae 
accomplishment  of  assigned  operational  missions.  The  planned  DPS  capabilities 
and  efficiencies  should  reduce  the  number  of  backlogged  MC&G  products.  The 
need  for  a  system  to  quickly  and  accurately  produce  MC&G  products  was 
demonstrated  by  the  U.  S.  involvement  in  Operation  Desert  Storm  in  Southwest 
Asia.  The  United  States  may  be  required  to  intervene  in  other  areas  of  the 
world  where  more  rugged  terrain  would  provide  cartographers  many  more 
challenges  than  the  desert  terrain  in  Southwest  Asia.  DoD  must  have  an 
effective  digital  MC&G  production  system  to  help  ensure  that  all  military 
objectives  can  be  effectively  achieved. 

Purpose  of  Milestone  IV  Review.  A  Defense  Acquisition  Board  Milestone  IV 
review  of  the  DPS  would  assess  all  aspects  of  the  program  to  verify  that  DMA 
has  an  effective  development  strategy  and  that  the  program  meets  planned 
objectives  as  development  is  finalized  and  major  modifications  are  made.  The 
review  should  determine  the  status  of  DPS  software  and  establish  criteria  for  the 
DPS  to  formally  enter  the  Operations  and  Support  phase  and  for  DMA  to 
proceed  with  modifying  the  DPS.  The  security  level  of  the  program  has  been 
reduced  and  the  Director,  Systems  Center,  intended  to  use  the  development 
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process  prescribed  by  DoD  Directive  5000.1  for  the  modifications  to  the  DPS. 
The  Defense  Acquisition  Board  review  would  ensure  DMA  complied  with  the 
provisions  of  DoD  Directive  5000.1. 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications  and 
Intelligence)  Expertise  in  MC&G.  DoD  Directive  5137.1,  "Assistant 
Secretary  of  Defense  (Command,  Control,  Communications  and  Intelligence) 
(ASD(C%)),"  February  12,  1992,  states  that  the  Assistant  Secretary  of  Defense 
(Command,  Control,  Communications  and  Intelligence)  is  responsible  for 
exercising  direction,  authority,  and  control  over  the  DMA.  However,  the  staff 
of  the  Assistant  Secretary  of  Defense  (Command,  Control,  Communications  and 
Intelligence)  lacks  sufficient  expertise  in  MC&G  matters.  To  provide  the 
necessary  expertise,  the  DMA  assigned  a  liaison  officer  to  the  Assistant 
Secretary.  The  liaison  officer  worked  on  the  Assistant  Secretary's  staff,  but  an 
official  in  DMA  Headquarters  prepared  the  officer's  performance  evaluation. 
The  liaison  officer  is  not  a  top-level  decision  maker  in  the  Assistant  Secretary's 
office  or  in  the  DMA.  However,  as  a  staff  officer,  the  individual  analyzes 
courses  of  action  and  makes  recommendations.  Working  as  a  staff  officer  for 
the  Assistant  Secretary  and  being  evaluated  by  a  DMA  official  creates  a 
potential  conflict  of  interest  for  the  individual.  Although  the  perception  of  a 
conflict  of  interest  exists,  the  audit  did  not  identify  any  instances  in  which  the 
integrity  of  the  liaison  officer  was  compromised.  Independent  oversight  by  the 
Assistant  Secretary,  as  required  by  DoD  Directive  5137.1,  would  prevent  any 
actual  or  perceived  conflict  of  interest. 


Conclusion 


An  effective  digital  MC&G  production  system  will  have  significant  utility  and  is 
needed  by  the  DoD.  The  development  of  the  DPS  to  date  has  been  a  complex 
task  and  is  the  result  of  hard  work  by  many  dedicated  Government  and 
contractor  personnel.  Further,  the  DMA  plans  to  modify  the  DPS  to  meet  new 
customer  requirements  and  to  produce  new  products.  The  DPS  is  being  used 
for  production,  but  the  proof  of  the  system  will  be  completed  products.  Given 
the  size,  complexity,  cost,  and  importance  of  the  DPS,  more  intensive  oversight 
of  the  DPS  is  warranted. 


Recommendations,  Management  Comments,  and  Audit 
Response 


1.  We  recommend  that  the  Under  Secretary  of  Defense  for  Acquisition  and 
Technology  conduct  a  Defense  Acquisition  Board  Milestone  IV  review  of 
the  current  status  and  planned  modifications  of  the  Digital  Production 
System. 
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2.  We  recommend  that  the  Assistant  Secretary  of  Defense  (Command, 
Control,  Communications  and  Intelligence)  provide  independent  oversight 
of  mapping,  charting,  and  geodesy  activities  at  the  Defense  Mapping 
Agency  as  required  by  DoD  Directive  5137.1,  "Assistant  Secretary  of 
Defense  (Command,  Control,  Communications  and  Intelligence) 
(ASDiC^D),"  February  12,  1992. 

Management  Comments  on  the  Recommendations.  The  Assistant  Secretary 
of  Defense  (Command,  Control,  Communications  and  Intelligence)  partially 
concurred  in  the  recommendations  to  perform  a  Milestone  IV  review  and  to 
provide  independent  oversight  of  the  Digital  Production  System.  The  Assistant 
Secretary  disagreed  that  a  Milestone  IV  review  should  be  conducted,  but  stated 
that  the  Deputy  Assistant  Secretary  of  Defense  (Command,  Control, 
Communications  and  Intelligence  Acquisition)  would  conduct  an  in-process 
review  of  the  DPS.  The  Assistant  Secretary  stated  that  the  puipose  of  the 
in-process  review  was  to  assess  the  status  of  the  DPS,  to  review  DMA's 
strategic  plan  for  the  DPS,  and  to  determine  intervals  for  fiiture  in-process 
reviews. 

Audit  Response.  The  Assistant  Secretary's  comments  meet  the  intent  of  the 
recommendations,  and  no  further  comments  are  required. 
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Appendix  A.  Description  of  Digital  Production 

System  Segments 


The  DPS  consists  of  seven  segments.  Generally,  production  work  cannot  begin 
in  one  segment  vmtil  work  has  been  completed  in  previous  segments.  The 
functions  of  the  seven  segments  are  briefly  described  below. 

Data  Services.  This  segment  transfers  data  among  the  segments  and  archives 
MC&G  data  extracted  from  digital  imagery.  Data  Services  uses 
five  programming  languages  and  has  about  911,000  lines  of  software  code. 
This  segment  was  developed  by  the  Hughes  Corporation. 

Production  Management.  This  segment  provides  a  single  integrated 
production  management  capability  for  all  DMA  production  systems.  Production 
Management  schedules,  manages,  and  monitors  production  resources  and 
assignments.  Production  Management  uses  four  programming  languages  and 
contains  about  1.4  million  lines  of  software  code.  The  contractor  for 
Production  Management  was  General  Electric  Aerospace  Division,  a  division  of 
the  General  Electric  Company,  that  was  bought  by  Martin  Marietta  Corporation 
in  1993.  The  Systems  Center  transferred  about  750,000  lines  of  software  code 
from  the  MARK  85  Data  Integration  segment  to  Production  Management. 

Source  Acquisition.  This  segment  controls  DMA's  imagery  data  and  asks 
external  agencies  to  collect  digital  imagery  for  the  DPS.  Source  Acquisition 
was  developed  for  the  MARK  85  system  and  is  now  used  in  the  DPS.  Source 
Acquisition  contains  558,000  lines  of  software  code  and  uses  four  programming 
languages.  General  Electric  Aerospace  Division  (now  a  component  of  Martin 
Marietta  Corporation)  developed  the  Source  Acquisition  segment. 

Source  Preparation.  This  segment  evaluates  source  materials  and  prepares 
production  feasibility  studies.  In  combination  with  Hardcopy  Exploitation, 
another  DPS  segment.  Source  Preparation  geopositions  the  digital  imagery. 
Geopositioning  matches  points  or  areas  on  the  imagery  to  points  on  the  earth’s 
surface.  In  addition.  Source  Preparation  produces  Point  Positioning  products. 
Source  Preparation  uses  four  programming  languages  and  contains  about 
1.4  million  lines  of  software  code.  E-Systems,  Inc.,  developed  the  Source 
Preparation  segment. 

Hardcopy  Exploitation.  This  segment  performs  final  steps  in  geopositioning 
the  imagery.  DBA  Systems  developed  the  Hardcopy  Exploitation  segment 
originally  for  the  MARK  85.  Hardcopy  Exploitation  contains  about 
353,000  lines  of  software  code  and  uses  one  programming  language. 

Data  Extraction.  This  segment  uses  two  images  simultaneously  to  extract 
feature  and  elevation  data  from  digital  imagery.  Dual  data  extraction  enables 
heights  of  features  and  elevations  of  terrain  to  be  identified.  Data  Extraction 
performs  quality  control  checks  to  ensure  extracted  data  are  in  compliance  with 
extraction  specifications.  In  addition.  Data  Extraction  produces  Terrain 
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Contour  Matching  Terminal  Matrix  products.  The  General  Dynamics 
Corporation  developed  this  segment,  which  uses  about  900,000  lines  of  software 
code  and  four  programming  languages. 

Product  Generation.  Except  for  Point  Positioning  Products  and  the  Terrain 
Contour  Matching  Terminal  Matrix  product,  all  product  finishing  work  is 
performed  in  this  segment.  Product  Generation  performs  quality  control  checks 
to  ensure  production  is  in  compliance  with  product  specifications.  When 
heights  of  features  and  elevations  of  terrain  are  not  required.  Product 
Generation  uses  one  image  to  extract  feature  data  from  digital  imagery  and 
cartographic  sources.  Product  Generation  uses  one  programming  language  and 
is  the  largest  segment  with  about  2.2  million  lines  of  software  code.  The 
Intergraph  Corporation  developed  this  segment. 
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The  Digital  Production  System  is  designed  to  produce  24  MC&G  products. 

Aeronautical  Products 

Flight  Information  Publications 
Tactical  Pilotage  Chart 
Operational  Navigation  Chart 
Joint  Operations  Graphic  (Air) 

Joint  Operations  Graphic  (Radar) 

U.S.  Air  Target  Chart  (1:200,000) 

Point  Positioning  Programs 
Point  Positioning  Data  Base 

Hydrographic  Products 

Harbor  and  Approach  Chart 
Coastal  Chart 
Combat  Chart 

Topographic  Products 

1:50,000  Topographic  Line  Map 
Joint  Operations  Graphic  (Ground) 

1:50,000  Tactical  Terrain  Analysis  Data  Base^ 

1:250,000  Planning  Terrain  Analysis  Data  Base^ 

Digital  Products 

Automated  Air  Facilities  Information  File 
Digital  Feature  Analysis  Data,  Level  I 
Digital  Feature  Analysis  Data,  Level  11 
Digital  Terrain  Elevation  Data,  Level  I 
Digital  Terrain  Elevation  Data,  Level  IF 
TERCOM^  Landfall  Matrix 
TERCOM  Enroute/Midcourse  Matrix 
TERCOM  Terminal  Matrk 
Vertical  Obstruction  Data^ 


^The  product  is  no  longer  needed  because  customers  terminated  their  requirements. 
The  product  will  be  replaced  by  Tactical  Terrain  Data  in  FY  1996. 

^Terrain  Contour  Matching. 
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Joint  Operations  Graphic  (Ground) 
1:50, 0()0  Tactical  Terrain  Analysis 
Data  Base 

1:250,000  Planning  Terrain  Analysis 
Data  Base 


Development  Contracts  Critical  Design  Review  DMA*s  Product  Baseline 

(May  1984)  (December  1988)  (September  1989) 

28  Products  31  Products  24  Products 


Appendix  C.  Changes  to  Digital  Production  System  Products 


VO  VO  VO  NO 

I  I  I  I  I 
I  I  t  I  I 


VO  VO 

X  X  X  X  X 


X  X  X  X  X 


X  X  X  X  X 


§ 

S 

Iti 

fill 

CO  ^  wa 

^  S  2 

^  sa 


-o 

» tr>  O  »n 
I  CNi  »n  cs 


X  X  X  X  X 


B  B 

a  d 

Q  Q 
§3  .S3  .S3 

CO  CO 

a  -I*  •I' 

'Is!  I 


css  a 

l'P  =  ^ 


.15  o  'S  S  -ti 
.2f  3  S  .5?  J  .2? 
Q  <  O  O 


<  W 

I  I 

l-H  S  ^  ID 

rn  ^ 

’3  'Td  *3  rij 
>  2  >  ™ 

Q  Q 


-{"as: 

O  H( 


.*2  VO 
■H  ^ 

§  O  ^ 

U  <u  >H 

.3  5^  c 

'3  ,B^B^ 

S  I  g  QQ 

^  §  g  g  g 

5  2  ^  .tJ 

O  -S?  .2 

o,  W)  ^  a>  3 

o  ‘3  ^  2  'S  ^ 

*2  .2  .y  o  2  .3 

i2  .5  S  ^  c5 

3  25  y  H  »-i  2 

S  o  d  Qj  o  3 

S 

E  ■  S  «  2  S  is 

ISfga  «  §8  2 

.2  2  ^  I  e*^  2 

«3  ’Hi’O  *0  S  *-•  w  ^ 
R  So  g  o  23  fl' 

I  ^  -3  2  -a 


3  O  J  O  S 

— •cScnTi-vnvor^oo 


26 


Appendix  D.  Summary  of  DoD  Hotline 

Allegations 


We  made  the  audit  as  a  result  of  two  anonymous  DoD  Hotline  complaints 
containing  25  allegations  related  to  DMA  and  DPS.  One  of  the  Hotline 
complaints  contained  allegations  regarding  the  overall  management  of  the  DMA 
and  the  funding  and  contracting  of  DMA  projects.  The  second  Hotline 
complaint  alleged  improprieties  in  the  fimding,  contracting,  development,  and 
operation  of  Ae  DPS.  DoD  Hotline  officials  referred  four  allegations  either  to 
Departmental  Inquiries,  Inspector  General,  DoD,  or  Inspector  General,  DMA, 
for  review.  In  addition,  the  Inspector  General,  DoD,  referred  one  allegation  to 
the  Defense  Criminal  Investigative  Service  for  investigation.  The  remaining 
20  allegations  and  the  audit  results  are  summarized  in  the  following  paragraphs. 
Based  on  the  audit  results,  allegations  2,  3,  4,  7,  and  10  relate  to  deficiencies  in 
DMA's  development  of  the  DPS  and  are  discussed  in  Part  II  of  the  report. 

1.  Allegation.  DMA  was  managing  the  DPS  as  a  classified  program,  even 
though  a  great  deal  of  the  program  was  not  classified. 

Audit  Result.  The  allegation  was  substantiated.  In  June  1989,  the  relationship 
between  DMA  and  the  DPS  contractors  was  declassified.  Additional  steps 
could  be  taken  to  reduce  the  security  level  of  most  future  contracts.  On  April  1, 
1994,  the  Inspector  General,  DoD,  issued  Audit  Report  No.  94-076, 
"Contracting  Authority  at  the  Defense  Mapping  Agency,"  on  DPS  contracting 
that  will  result  in  more  contracts  that  do  not  require  special  security  controls.  In 
response,  the  DMA  began  developing  a  plan  to  transition  contracting  from  a 
non-DoD  agency  to  DMA.  Certain  aspects  of  the  program  remain  classified, 
and  personnel  must  have  appropriate  security  access  to  sensitive  compartmented 
information  in  developing  and  operating  the  DPS. 

2.  Allegation.  DoD  directives  and  instructions  were  not  followed,  and  the 
program  had  not  been  formally  reviewed  by  external  decision  makers.  Without 
baseline  agreements  and  external  reviews,  the  DPS  can  be  whatever  the  Systems 
Center  says  it  is. 

Audit  Result.  The  allegation  was  partly  substantiated  in  that  regulations  were 
not  followed  and  an  external  review  was  not  done.  The  Assistant  Secretary  of 
Defense  (Command,  Control,  Communications  and  Intelligence)  did  not  require 
DMA  to  comply  with  DoD  acquisition  policies  because  non-DoD  channels  and 
practices  were  used  in  the  DPS  development.  The  DMA  identified  specific 
products  for  production  on  the  DPS  and  changed  the  products  as  customer 
requirements  changed.  The  development  of  the  DPS  and  the  lack  of  external 
reviews  are  discussed  in  Part  II,  Finding  B. 

3.  Allegation.  Little  program  documentation  exists  to  trace  the  requirements 
for  the  DPS  or  for  its  milestones. 

Audit  Result.  The  allegation  was  partly  substantiated  in  that  configuration 
management  procedures  needed  improving.  The  Systems  Center  developed 
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system  and  product  specifications  and  interface  control  documents  that  identified 
the  requirements  for  the  DPS.  In  addition,  the  Systems  Center  prepared 
detailed  plans  for  DPS  development.  We  reviewed  the  hierarchy  of 
specifications  and  traced  milestone  definitions  in  program  documents  for  initial 
operational  capabihty  and  full  operational  capability  to  the  original  documents. 
Good  configuration  management  procedures  would  help  ensure  sufficient  and 
consistent  documentation.  The  audit  results  regarding  configuration 
management  are  discussed  in  Part  II,  Finding  A. 

4.  Allegation.  The  DPS  does  not  work  as  originally  planned.  Original 
requirements  that  could  not  be  met  cannot  be  traced. 

Audit  Result.  The  allegation  was  partly  substantiated  because  specifications 
changed  as  the  DPS  was  developed  and  used.  However,  the  basic  objective  of 
using  digital  imagery  instead  of  film  to  produce  MC&G  products  remained  the 
same.  The  audit  showed  that  the  Systems  Center  could  track  requirements. 
Based  on  the  results  of  1992  reviews  of  completed  system  segments,  the 
Systems  Center  determined  that  except  for  162  of  4,595  segment  and  segment 
interface  specifications,  the  software  developed  by  the  contractors  met  contract 
specifications.  DMA's  Executive  Confi^ration  Control  Board  approved 
waivers  for  112  of  the  162  unmet  specifications  because  the  Board  considered 
delivered  capability  to  be  sufficient.  As  a  result  of  additional  tests  and 
corrections  of  software  defects,  the  Systems  Center  considered  49  of  the 
remaining  50  unmet  specifications  to  have  been  met  by  September  30,  1993. 
As  of  March  10,  1994,  the  Systems  Center  was  completing  a  waiver  for  the 
final  unmet  specification.  As  the  DPS  software  has  been  used  in  operational 
testing  and  production,  DMA  has  identified  software  errors  and  needed 
modifications.  Software  development  for  the  Flight  Information  Publications 
and  Automated  Air  Facilities  Information  File  products  and  software 
discrepancy  reports  are  discussed  in  detail  in  Part  II,  Finding  A. 

5.  Allegation.  The  DPS  could  not  replicate  the  criteria  for  initial  operational 
capability  after  the  DMA  declared  that  the  criteria  had  been  met. 

Audit  Result.  The  allegation  was  substantiated,  but  the  audit  identified  no 
adverse  consequences.  DMA  determined  that  to  declare  initial  operational 
capability,  Ae  system  would  have  to  be  capable  of  producing  4  of  the  24  DPS 
products.  According  to  the  system  integration  contractor's  "Final  Report  for 
Demonstration  909:  Single  Product  Operations  (SPO)  Demonstration,"  May  28, 
1991,  which  contains  the  results  of  initial  operational  capability  testing,  all 
four  products  were  completed  by  March  23,  1991.  DMA  declared  initial 
operational  capability  on  March  27,  1991.  Systems  Center  personnel  stated  that 
they  changed  the  system  shortly  after  DMA  declared  initial  operational 
capability.  The  changes  prevented  the  system  from  operating  in  the  same 
manner  as  when  initial  operational  capability  was  declared.  The  Systems  Center 
made  the  changes  to  fix  problems  identified  during  the  production  of  the 
products  for  the  initial  operational  capability  milestone  and  to  further  enhance 
the  system.  The  Systems  Center  continued  to  develop  and  enhance  the  DPS 
during  FY  1991  and  FY  1992  in  preparation  for  initiating  production 
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operations.  The  DMA  did  not  use  the  DPS  to  initiate  the  production  of  MC&G 
products  until  July  1992,  when  geopositioning  of  imagery  began  in  the  Source 
Preparation  segment. 

6.  Allegation.  The  definition  of  full  operational  capability  was  not  consistent 
throughout  the  program  development. 

Audit  Result.  The  allegation  was  substantiated.  Criteria  were  inconsistent  in 
the  DPS  System  Specification  and  the  DPS  Transition  Integration  Plan  for 
deciding  when  full  operational  capability  of  the  DPS  had  been  achieved.  The 
DPS  System  Specification  E9000B,  September  13,  1984,  as  revised  on 
September  6,  1989,  defined  full  operational  capability  as  follows.  "At  Final 
Operational  Capability  (FOC-E4.5)  the  MARK  90  System  will  have  sufficient 
capability  to  produce  100  percent  of  the  DMA  production  allocated  to  it."  The 
DPS  Transition  Integration  Plan,  June  15,  1990,  as  revised  on  June  12,  1991, 
defined  full  operational  capability  as  follows.  "At  Full  Operational  Capability 
(FOC),  the  MARK  90  System  is  in  an  operational  condition,  with  the  capability 
to  produce  the  assigned  customer-releasable  products."  Although  that  criteria 
had  been  approved  by  DMA's  Executive  Configuration  Control  Board,  Systems 
Center  development  officials  stated  that  they  based  the  declaration  of  full 
operational  capability  on  the  completion  of  certain  operational  tests.  The 
determination  of  full  operational  capability  by  Systems  Center  officials  is 
discussed  below. 

7.  Allegation.  Development  of  the  program  was  tied  to  milestone  dates  rather 
than  to  milestone  criteria. 

Audit  Result.  The  allegation  was  partly  substantiated.  When  Systems  Center 
officials  determined  full  operational  capability  was  achieved,  the  DPS  could  not 
produce  the  Flight  Information  Publications  and  Automated  Air  Facilities 
Information  File  products.  The  software  for  the  Flight  Information  Publications 
and  Automated  Air  Facilities  Information  File  products  was  not  included  in  the 
tests  for  declaring  full  operational  capability  and  did  not  pass  all  acceptance  tests 
until  December  1993.  In  addition,  the  Digital  Feature  Analysis  Data,  Level  II, 
product  produced  for  declaring  full  operational  capability  was  not  customer- 
releasable  because  it  did  not  meet  customer  requirements.  Also,  as  of 
September  9,  1993,  DMA  had  not  demonstrated  that  the  DPS  was  capable  of 
producing  7  of  24  scheduled  products  by  producing  them  either  in  testing  or  in 
production  operations.  Those  seven  products  are  discussed  further  in  Part  II, 
Finding  B. 

The  date  for  full  operational  capability  changed  as  the  DPS  was  developed.  In 
FY  1986,  as  a  result  of  DoD  budget  reductions,  DMA  moved  the  scheduled  full 
operational  capability  date  from  September  1991  to  March  1992.  In  FY  1989, 
as  a  result  of  the  Red  Team's  review,  discussed  in  Part  II,  Finding  B,  DMA 
again  changed  the  scheduled  full  operational  capability  date  from  March  1992  to 
November  1992.  Schedules  for  DPS  tests  slipped  after  1989,  but  DMA  did  not 
change  the  full  operational  capability  date  again  or  establish  incremental  full 
operational  capability  dates  for  DPS  operations  as  individual  products  were 
developed  and  produced.  DMA  officials  informed  us  that  the  full  operational 
capability  date  was  not  considered  important  to  DPS  development  because  more 
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cartographers  would  transfer  to  the  DPS  system  and  the  number  of  products 
being  produced  would  increase  through  FY  1995.  DMA  officials  told  us  that 
they  wanted  to  develop  the  system  on  time  and  on  budget  and  that  they  had  done 
so. 

DMA  officials  stated  that  they  based  the  declaration  of  full  operational 
capability  on  the  completion  of  Demonstration  (Demo)  910.  Demo  910  was  the 
last  of  a  series  of  tests  culminating  in  tests  of  all  DPS  segments.  During 
Demo  910,  8  of  24  DPS  products  were  tested  and  produced.  DPS  integration 
officials  stated  that  in  1989,  an  internal  Tiger  Team  identified  the  24  products  to 
be  produced  by  the  DPS  and  the  8  products  to  be  tested  during  Demo  910.  The 
eight  products  tested  during  Demo  910  were  the: 

o  1:50,000  Topographic  Line  Map; 

o  Joint  Operations  Graphic  (Ground); 

o  Coastal  Chart; 

o  TERCOM*  Landfall  Matrix; 

o  TERCOM  Enroute/Midcourse  Matrix; 

o  TERCOM  Terminal  Matrix; 

o  Digital  Feature  Analysis  Data,  Level  11;  and 

o  Digital  Terrain  Elevation  Data,  Level  1. 

The  Tiger  Team  selected  the  eight  products  for  testing  because  engineering 
analysis  determined  that  the  software  in  those  eight  products  represented  the 
most  important  and  frequently  used  software  in  the  DPS.  Systems  Center 
officials  believed  that  sufficient  time  was  not  available  in  the  program  schedule 
to  test  all  24  products,  and  they  wanted  to  maximize  use  of  the  limited  available 
time.  The  last  of  the  eight  products  in  Demo  910  was  completed  on 
October  31,  1992,  and  DMA  declared  full  operational  capability  on 
November  5,  1992. 

We  believe  Systems  Center  officials  did  not  have  a  sound  basis  to  determine  that 
full  operational  capability  had  been  met  as  of  November  5,  1992.  Demo  910 
testing  was  based  on  a  limited  set  of  cartographic  data  and  did  not  assure  that 
the  DPS  was  capable  of  producing  all  products.  Although  the  declaration  of  full 
operational  capability  may  have  been  premature,  we  could  not  identify  any 
adverse  consequences.  As  discussed  in  Part  n.  Finding  B,  the  DPS  was  not 
formally  reviewed  by  external  DoD  decision  makers  and  the  DMA  was  allowed 
to  use  its  own  tailored  development  process  (see  Appendix  E).  As  discussed  in 
Allegation  8,  the  declaration  did  not  affect  the  funding  appropriation  used  for 
the  development  of  the  DPS. 


*Terrain  Contour  Matching 
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8.  Allegation.  The  DPS  would  be  declared  fully  operational  capable  in 
November  1992  in  order  to  use  Operation  and  Maintenance  funds  to  fix  the 
DPS. 

Audit  Result.  The  allegation  was  not  substantiated.  DMA's  decision  to  cease 
using  the  Research,  Development,  Test,  and  Evaluation  appropriation  to  fund 
DPS  development  was  based  on  segment  completion  reviews  rather  than  on  the 
declaration  of  full  operational  capability.  Segment  completion  reviews, 
conducted  from  May  to  September  1992,  determined  that  the  segments  were 
ready  to  proceed  to  system  level  tests.  DMA  had  budgeted  Research, 
Development,  Test,  and  Evaluation  funds  for  the  development  of  the  DPS 
through  September  1992,  the  end  of  FY  1992.  The  date  for  fiill  operational 
capability  was  scheduled  for  and  declared  on  November  5,  1992.  DMA  has 
budgeted  for  modifications  to  the  DPS  using  Research,  Development,  Test,  and 
Evaluation  funds  through  FY  1999. 

9.  Allegation.  Modifications  to  the  system  were  identified  as  preplanned 
product  improvements  when  the  modifications  were  not  plaimed  as  part  of  the 
original  program.  Calling  the  modifications  preplanned  improvements  is  a  way 
to  hide  problems. 

Audit  Result.  The  allegation  was  not  substantiated,  although  specifications  and 
the  products  to  be  produced  on  the  DPS  changed  as  the  DPS  was  developed. 
The  evolution  of  the  products  to  be  produced  by  the  DPS  is  summarized  in 
Appendix  C.  Modifications  to  the  DPS  are  discussed  in  Part  II,  Finding  A. 

10.  Allegation.  DMA  lacks  effective  mid-  to  long-term  planning.  The  lack  of 
planning  has  caused  unnecessary  overtime  and  contracting  of  work. 

Audit  Result.  The  allegation  was  substantiated  regarding  the  development  of 
the  Flight  Information  ^blications  and  Automated  Air  Facilities  Information 
File  products  and  regarding  program  planning  for  the  modification  of  the  DPS. 
The  audit  results  are  discussed  in  Part  II,  Finding  A.  The  adequacy  of  planning 
for  all  of  DMA  will  be  reviewed  as  part  of  an  FY  1995  Inspector  General, 
DoD,  inspection  of  DMA.  In  addition,  DMA's  plan  to  standardize  digital 
MC&G  data  used  in  DoD  systems  will  be  audited  by  the  Inspector  General, 
DoD,  during  FY  1994. 

11.  Allegation.  Senior  DMA  officials  were  not  aware  of  the  status  of  the  DPS 
development  because  status  reports,  such  as  the  quarterly  report  submitted  to  the 
Director,  DMA,  on  April  1,  1992,  were  misleading. 

Audit  Result.  The  allegation  was  partly  substantiated  in  that  the  quarterly 
reports  were  general  in  nature.  During  FY  1992,  the  Directors  of  DMA 
components  submitted  their  quarterly  reports  to  the  Director,  DMA.  Because 
the  reports  did  not  meet  the  intended  purpose  of  focusing  on  significant  issues, 
the  former  Director,  DMA,  canceled  the  reporting  requirement  on 
September  22,  1992.  However,  Systems  Center  officials  gave  the  former 
Director,  DMA,  a  monthly  briefing  on  the  status  of  the  DPS  development. 
Based  on  our  review  of  the  briefing  charts  used  at  those  meetings  and  on  our 
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interviews  with  DMA  senior  officials  and  the  former  Director,  DMA, 
substantive  development  issues  were  briefed  and  the  former  Director,  DMA, 
was  informed  of  significant  problems  that  occurred. 

12.  Allegation.  All  the  extra  hours  worked  by  DMA  employees  to  make  the 
system  work  were  not  reported  and  were  not  recognized  in  the  cost  of  the  DPS. 

Audit  Result.  The  allegation  was  substantiated,  but  DoD  does  not  require  that 
costs  attributed  to  Government  employees  be  included  in  the  acquisition  costs  of 
major  systems.  Based  on  discussions  with  DMA  personnel,  many  DMA 
employees  worked  many  extra  hours  and  were  dedicated  to  the  successfiil 
development  of  the  DPS. 

13.  Allegation.  Non-DPS  projects  were  put  on  DPS  contracts  in  FY  1992. 
The  non-DPS  projects  were  outside  the  scope  of  the  DPS  contracts,  and  the 
non-DPS  projects  were  not  funded  from  the  same  budget  program  element  as 
the  DPS  projects.  Putting  projects  funded  by  different  program  elements  on  the 
same  contract  was  inappropriate. 

Audit  Result.  The  allegation  was  partly  substantiated.  The  Systems  Center 
placed  "non-DPS"  projects  on  DPS  development  contracts  because  the 
capabilities  being  developed  would  be  part  of  the  DPS.  The  Systems  Center 
differentiated  between  "non-DPS"  and  DPS  projects  only  to  distinguish  between 
enhancements  to  the  DPS  and  its  initial  development  capability. 

The  audit  identified  47  "non-DPS"  projects  that  included  53  tasks  that  were  to 
be  completed  from  April  through  October  1992.  We  judgmentally  selected  7  of 
the  53  tasks  to  determine  how  the  tasks  were  contracted  and  whether  the  funds 
involved  were  used  for  DPS  enhancements  or  for  system  development.  Four  of 
the  seven  tasks  were  placed  on  existing  DPS  development  contracts,  one  task 
was  placed  on  a  new  contract,  and  two  tasks  were  placed  on  other  existing 
contracts.  Based  on  our  interviews  with  Systems  Center  officials  and  reviews  of 
contract  files,  the  audit  determined  that  the  four  tasks  placed  on  existing  DPS 
contracts  were  for  capabilities  that  would  be  used  in  the  DPS  and  were  within 
the  scope  of  the  DPS  contracts. 

Regarding  funding,  we  determined  that  "non-DPS"  and  DPS  projects  were 
funded  under  different  budget  line  numbers,  but  both  line  numbers  were  within 
budget  program  element  3051 39B.  Within  certain  limits,  the  DMA  was 
authorized  to  transfer  funds  between  projects  within  the  same  budget  program 
element.  Generally,  transfers  of  less  than  $4  million  or  new  projects  costing 
less  than  $2  million  had  to  be  approved  by  DMA's  Executive  Configuration 
Control  Board  and  Headquarters,  DMA.  Transfers  of  more  than  $4  million  or 
new  projects  costing  more  than  $2  million  had  to  be  approved  by  the  Under 
Secretary  of  Defense  (Comptroller)  and  appropriate  congressional  committees. 
DPS  program  documents  showed  that  the  funds  for  the  seven  "non-DPS"  tasks 
were  used  for  development  of  the  capability  specified  in  the  tasking  or  were 
reallocated  to  other  "non-DPS"  projects.  The  audit  concluded  that  no 
improprieties  occurred  in  putting  the  "non-DPS"  projects  on  the  DPS  contracts. 
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14.  Allegation.  A  contracting  officer's  technical  representative  (COTR)  held 
unauthorized  discussions  with  a  contractor  on  a  proposed  contract,  and 
negotiations  were  concluded  shortly  after  the  discussions  were  held. 

Audit  Result.  The  allegation  was  not  substantiated.  The  audit  fotmd  no 
evidence  that  the  COTR  had  violated  his  position  of  trust.  COTRs  hold 
discussions  wiA  contractors  regarding  contract  proposals  to  ensure  that  bids 
represent  the  work  desired  by  DMA  and  to  obtain  additional  information 
regarding  the  details  of  the  cost  proposal.  The  contracting  officer  and 
contracting  specialists  knew  of  no  instance  in  which  a  COTR  had  conducted  a 
fact-finding  session  with  a  contractor  without  the  knowledge  of  the  contracting 
officer  or  specialist. 

15.  Allegation.  The  COTRs  gave  direction  to  the  contractors  and  obligated  the 
Government. 

Audit  Result.  The  allegation  was  not  substantiated  regarding  COTR  actions 
since  1991.  In  1991,  the  Systems  Center  determined  that  a  COTR  had  taken 
action  that  could  have  obligated  the  Government.  Because  of  that  action,  the 
contracting  officer  issued  a  memorandum,  "Unauthorized  Commitments," 
September  6,  1991,  to  Systems  Center  personnel,  and  a  message,  "Unauthorized 
Commitments,"  September  10,  1991,  to  contractors,  emphasizing  that  only  the 
contracting  officer  can  obligate  the  Government.  Also,  the  contracting  officers 
require  COTRs  to  sign  a  statement  acknowledging  their  responsibilities  and  lack 
of  authority  to  obligate  the  Government.  The  audit  found  no  evidence  that 
COTRs  had  given  inappropriate  directions  to  the  contractors  or  had  taken 
actions  that  would  have  obligated  the  Government  since  1991. 

16.  Allegation.  The  COTRs  are  relying  on  rough  order  of  magnitude  estimates 
submitted  by  contractors  without  verifying  the  accuracy  of  the  estimates. 

Audit  Result.  The  allegation  was  substantiated,  but  the  audit  identified  no 
impropriety.  Contractors  submit  rough  order  of  magnitude  cost  estimates  to 
DMA  on  system  changes  proposed  by  either  the  contractors  or  DMA.  The 
Executive  Coiffiguration  Control  Board  uses  the  estimates  in  determining 
whether  the  proposed  change  is  needed  and  affordable  to  pursue  negotiations 
with  the  contractor.  The  rough  orders  of  magnitude  lack  sufficient  detail  to  be 
analyzed  for  accuracy.  After  the  Executive  Configuration  Control  Board 
determines  that  the  proposed  change  is  needed  and  affordable,  the  contractor 
submits  detailed  cost  estimates  to  the  contracting  officer  for  analysis  by  the 
COTRs.  If  the  proposed  cost  exceeds  $1  million,  the  contracting  officer  also 
sends  the  estimates  to  the  Defense  Contract  Audit  Agency  for  analysis.  For 
some  tasks,  the  contractor  is  allowed  to  begin  work  before  negotiations  are 
completed,  but  the  Government's  liability  for  the  work  performed  by  the 
contractor  does  not  exceed  an  amount  specified  by  the  contracting  officer. 
Otiierwise,  contractual  obligations  are  not  made  until  negotiations  based  on 
detailed  cost  information  are  completed. 
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17.  Allegation.  Contracts  are  awarded  on  a  sole  source  basis  to  contractors. 

Audit  Result.  The  allegation  was  partly  substantiated  in  that  some  contracts 
have  been  awarded  on  a  sole  source  basis.  DMA  awarded  the  development 
contracts  for  the  seven  DPS  segments  on  a  competitive  basis  and  awarded 
contracts  for  other  work  regarding  the  DPS  on  both  a  competitive  and  a  sole 
source  basis.  The  audit  examined  the  justifications  for  seven  sole  source 
contracts  for  DPS  enhancements  or  system  support  that  were  awarded  during 
January  1990  through  May  1993.  The  audit  determined  that  the  award  of  the 
seven  contracts  on  a  sole  source  basis  was  justified. 

18.  Allegation.  The  prime  contractors  for  the  DPS  segments  are  also 

subcontractors  to  each  other. 

Audit  Result.  The  allegation  was  substantiated,  but  the  audit  found  no 
evidence  indicating  irregularities  in  the  relationships.  The  use  of  subcontractors 
is  a  business  decision  made  by  the  prime  contractor,  depending  on  a  comparison 
of  its  expertise  to  expertise  available  from  other  companies  at  a  cost-effective 
price. 

19.  Allegation.  DMA  has  contracts  with  three  contractors  to  provide 
engineering  support  for  the  DPS  when  one  contractor  would  suffice.  In 
addition,  engineering  support  contractors  were  doing  work  that  was  within  the 
scope  of  another  engineering  support  contractor. 

Audit  Result.  The  allegation  was  substantiated  in  that,  during  the  audit,  the 
Systems  Center  had  contracts  with  three  contractors  to  do  engineering  support 
work,  and  some  overlap  existed  in  the  contractors'  statements  of  work.  The 
Systems  Center  desired  the  overlap  because  in  some  cases,  the  expertise  of 
one  contractor  was  better  than  another  in  a  particular  technical  area.  The 
overlap  allowed  the  contractor  with  more  expertise  to  be  selected  to  perform  a 
specific  task.  The  contracts  with  the  engineering  support  contractors  were  cost 
or  time  and  materials  type  contracts.  In  cost  and  time  and  materials  contracts, 
the  contractor  is  paid  only  for  work  performed. 

20.  Allegation.  The  system  integration  contractor  has  a  potential  conflict  of 
interest  because  that  contractor  is  also  responsible  for  developing  DPS 
segments. 

Audit  Result.  The  allegation  was  substantiated  in  that  a  potential  conflict  of 
interest  existed,  but  the  audit  found  no  evidence  that  actual  conflicts  of  interest 
had  occurred.  The  potential  conflict  of  interest  was  recognized  by  both  DMA 
and  the  contractor.  General  Electric  Aerospace  Division,  (purchased  by  Martin 
Marietta  Corporation  in  1993).  General  Electric  Aerospace  and  Martin  Marietta 
established  organizational  and  procedural  safeguards  to  assure  the  independence 
of  the  system  integrator  function. 
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DMA's  strategy  for  the  development  of  the  DPS  evolved  as  DMA  gained 
experience  in  major  systems  development  and  better  understood  DPS 
requirements.  Because  of  the  complexity  and  unique  functionality  of  the  DPS, 
DMA  implemented  a  three-phase  approach  to  DPS  development.  The  three- 
phase  approach  resulted  primarily  from  DMA's  desire  to  develop  a  functional 
system  5iat  would  be  enhanced  later  and  from  the  need  to  develop  a  data  base  of 
digital  MC&G  data  before  products  could  be  produced. 

Phase  One-Developing  DPS  Core  Capability.  During  the  first  phase,  DMA 
developed  the  basic  DPS  capability.  Following  a  1988  review  by  a  panel  of 
software  experts  from  industry.  Government,  and  academia,  DMA  identified 
24  products  to  be  produced  by  the  DPS.  Development  of  the  capability  to 
produce  other  products  required  by  DMA  customers  was  deferred  until 
completion  of  &e  capability  to  produce  the  24  products.  DMA  officials 
determined  that  the  DPS  achieved  full  operational  capability  on  November  5, 
1992.  Those  officials  informed  the  Director,  DMA,  ^t  with  the  initiation  of 
full  operational  capability,  information  would  be  placed  in  the  MC&G  data  base 
and  that  finishing  of  products  would  start  later  in  FY  1993  as  data  became 
available  in  the  MC&G  data  base. 

Phase  Two-Entering  Data  in  a  Digital  Data  Base  and  Increasing  System 
Usage.  The  second  phase  was  to  enter  MC&G  data  in  a  digital  data  base  and 
increase  usage  of  the  system. 

Entering  MC&G  Data  into  a  Digital  Data  Base.  To  produce  finished 
products  on  the  DPS,  MC&G  data  must  be  in  a  digital  data  base.  Establishing  a 
digital  data  base  of  MC&G  data  encompasses  the  following  responsibilities. 

o  DMA  asks  external  agencies  to  collect  specific  digital  imagery. 

o  External  agencies  collect  digital  imagery  and  forward  it  to  DMA  with 
all  management  support  data. 

0  Cartographers  identify  the  location  of  the  digital  imagery  to  its 
position  on  the  earth,  extract  MC&G  data  from  the  digital  imagery,  and  store  it 
in  the  MC&G  digital  data  base. 

The  process  of  obtaining  digital  imagery  and  placing  information  in  the  MC&G 
data  base  can  take  more  than  1  year. 

Increasing  DPS  Usage.  DMA  planned  to  increase  both  DPS  usage  and 
production  through  FY  1995,  at  which  time  DMA  estimated  that  two  shifts 
would  be  working  on  all  segments  of  the  DPS  and  that  all  products  would  be  in 
production. 

Phase  Three-Enhancing  DPS  Capability.  The  third  phase  of  DPS 
development  involves  modifying  the  DPS  to  produce  additional  products.  The 
need  to  modify  die  production  capability  of  DPS  resulted  from  DMA's  decision 
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in  1989  to  defer  the  development  of  certain  products.  In  addition,  as  the  world 
order  changed  with  the  dissolution  of  the  Soviet  Union,  DMA  identified 
additional  modifications  needed  for  DPS  operations  and  capabilities.  Systems 
Center  personnel  developed  the  core  system  using  a  tailored  development 
process.  For  phase  three,  the  Director,  Systems  Center,  intends  to  use  the 
development  process  prescribed  by  DoD  Directive  5000. 1  as  discussed  in 
Part  II,  Finding  B. 
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Planned  Finishing  of  DPS  Products  During  FY  1994 
_ (As  of  February  18.  1994> _ 


Appendix  F.  FY  1994  Production  Program  for  the  Digital  Production  System 
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‘•Product  is  no  longer  needed  because  customers  deleted  their  requirements. 
^Terrain  Contour  Matching. 


Appendix  G 


Recommendation 

Reference 


A.l. 


A. 2.  and  A.3. 


A. 4.  and  A.5. 


A.6. 


B.l. 


B.2. 


.  Sunmiary  of  Potential  Benefits 
Resulting  from  the  Audit 


Description  of  Benefit  Type  of  Benefit 


Internal  Controls  and  Program 
Results.  Implementing  procedures 
to  identify  and  coordinate  customer 
requirements  early  in  system 
development  will  reduce  the  risk  of 
errors  and  unnecessary 
expenditures. 

Internal  Controls  and  Program 
Results.  Identifying  and  analyzing 
causes  for  problems  will  help 
prevent  system  errors. 

Internal  Controls  and  Program 
Results.  Assigning  personnel  to  the 
configuration  management  function 
and  developing  plans  for  the 
correction  of  Ae  configuration 
management  deficiencies  will 
improve  configuration  manajgement 
of  the  DPS  and  reduce  the  risk  of 
unnecessary  expenditures. 

Internal  Controls  and  Program 
Results.  Developing  a 
comprehensive  plan  for  the 
modifications  to  the  DPS  will 
reduce  the  risk  of  unnecessary 
expenditures. 

Program  Results.  Conducting  a 
Defense  Acquisition  Board 
Milestone  IV  review  of  the  DPS 
will  ensure  that  DPS  requirements 
are  documented  and  modifications 
are  well  planned. 

Compliance.  Providing  independent 
oversight  will  enhance  the  integrity 
of  the  oversight  of  the  DMA. 


Nonmonetary. 


Nonmonetary. 


Nonmonetary. 


Nonmonetary. 


Nonmonetary. 


Noiunonetary. 
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Appendix  H.  Organizations  Visited 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology,  Washington,  DC 
Defense  Systems  Management  College,  Fort  Belvoir,  VA 
Under  Secretary  of  Defense  (Comptroller),  Washington,  DC 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications  and  Intelligence), 
Washington,  DC 


Defense  Agency 

Defense  Mapping  Agency,  Fairfax,  VA 


Non-Government  Organizations 

Martin  Marietta  Corporation,  Valley  Forge,  PA 
Intergraph  Corporation,  Himtsville,  AL 
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Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology 
Under  Secretary  of  Defense  (Comptroller) 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications  and  Intelligence) 
Assistant  to  the  Secretary  of  Defense  (Intelligence  Oversight) 

Assistant  to  the  Secretary  of  Defense  (Public  Affairs) 


Department  of  the  Army 

Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Commander,  Naval  Facilities  Engineering  Command 
Auditor  General,  Department  of  die  Navy 


Department  of  the  Air  Force 

Auditor  General,  Department  of  the  Air  Force 


Defense  Agencies 

Director,  Defense  Contract  Audit  Agency 

Director,  Defense  Logistics  Agency 

Director,  Defense  Mapping  Agency 

Director,  National  Security  Agency 

Inspector  General,  Central  Imageiy  Office 

Inspector  General,  Defense  Intelligence  Agency 

Inspector  General,  National  Security  Agency 

Director,  Defense  Logistics  Studies  Information  Exchange 


Non-Defense  Federal  Organizations 

Office  of  Management  and  Budget 

U.S.  General  Accounting  Office,  National  Security  and  International  Affairs  Division, 
Technical  Information  Center 
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Non-Defense  Federal  Organizations  (cont'd) 


Chairman  and  Ranking  Minority  Member  of  Each  of  the  Following  Congressional 
Committees  and  Subcommittees 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 

Senate  Committee  on  Armed  Services 

Senate  Committee  on  Governmental  Affairs 

Senate  Select  Committee  on  Intelligence 

House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Services 
House  Committee  on  Government  Operations 

House  Subcommittee  on  Legislation  and  National  Security,  Committee  on  Government 
Operations 

House  Permanent  Select  Committee  on  Intelligence 

House  Subconunittee  on  Oversight  and  Evaluation,  House  Permanent  Select  Committee 
on  Intelligence 

House  Subcommittee  on  Program  and  Budget  Authorization,  House  Permanent  Select 
Committee  on  Intelligence 


Part  rv  -  Management  Comments 


Assistant  Secretary  of  Defense  (Command, 
Control,  Communications  and  Intelligence) 
Comments 


ASSISTANT  SECRETARY  OF  DEFENSE 

WASHINGTON.  D.C.  20301*3040 

October  13,  1994 


MEMORANDUM  FOR  INSPECTOR  GENERAL  OF  THE  DEPARTMENT  OF  DEFENSE 

SUBJECT:  Draft  DoD  Inspector  General  Audit  Report  on  the  Management  of  the  Digital 
Production  System  Development  at  the  Defense  Mapping  Agency 
(Project  No.  3RC-8008) 


The  Draft  Audit  Report  has  been  jointly  reviewed  and  the  Assistant  Secretary  of  Defense 
(Conunand,  Control,  Communications  and  Intelligence)  will  provide  the  single  coordinated 
response  to  your  findings  and  recommendations.  Our  specific  comments  are  set  forth  in  the 
attachment  to  this  memorandum. 

The  DoD  IG  report  raises  the  question  of  deficiencies  in  the  management  and  oversight  of 
the  Digital  Production  System  (DPS)  development.  While  we  agree  that  there  continues  to  be 
room  for  improvement  in  the  processes  and  methodologies  employed  to  manage  and  oversee  the 
development  of  a  system  as  complex  as  the  Digital  Production  System,  the  DoD  IG  did  not 
recognize  several  significant  accomplishments  and  ongoing  program  initiatives  to  address 
problem  areas  identified  by  the  Defense  Mapping  Agency  (DMA).  System  performance 
improvements  are  evident  as  a  result  of  Defense  Mapping  Agency’s  implementation  of 
procedures  to  identify  aeronautical  product  customer  requirements  prior  to  development,  the  use 
of  category  codes  to  identify  causes  of  problems  being  entered  in  the  Agency’s  Configuration 
Management  System,  and  the  use  of  software  measurements  and  metrics  analyses  to  manage 
change  to  the  baseline. 

We  agree  that,  without  adequately  trained  and  experienced  configuration  management 
personnel,  it  is  difficult  to  diagnose  the  causes  of  hardware  and  software  problems  impacting  the 
system.  Adequate  human  resources  now  exist  to  diagnose  and  resolve  system  performance 
problems-as  evidenced  by  numerous  instances  in  which  software  metrics  have  been  used  to 
identify,  analyze,  and  resolve  the  Digital  Production  System  performance  problems  at 
operational  sites.  Although  the  DoD  IG  reported  that  significant  software  defects  existed  in  the 
program’s  baseline,  the  DoD  IG  neglected  to  mention  that  the  Defense  Mapping  Agency  had 
already  taken  positive  steps  to  establish  and  implement  appropriate  procedures  to  analyze  the 
causes  of  trends  in  problems  identified  in  the  Digital  Production  System  discrepancy  reports  and 
to  take  corrective  action. 


COMMAND.  CONTROL. 
COMMUNICATIONS 
AND 

INTELLIGENCE 
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btelligence)  Conunents 


We  disagree  with  the  DoD  IG  determination  that  there  were  material  weaknesses  in  Defense 
Mapping  Agency’s  internal  material  control  processes  for  customer  requirements  identification, 
software  problem  analysis,  configuration  control,  and  Defense  Production  System  modification 
definition.  In  each  of  these  areas,  the  Defense  Mapping  Agency’s  internal  control  processes 
identified  the  problems  and  implemented  (or  were  implementing)  solutions. 

Finally,  we  point  out  that  the  Assistant  Secretary  of  Defense  (Command,  Control, 
Conununications  and  Intelligence)  consistent  with  the  statutory  authorities  contained  in  Public 
Law  96-51 1  and  DoD  Directive  5137.1.  has  overall  supervision  and  management  authorities  to 
act  as  the  DoD  Senior  Information  Management  Official  responsible  for  the  oversight  of  all  DoD 
information  management  programs  and  Federal  Information  Processing  resources;  and,  as  such, 
will  conduct  appropriate  independent  oversight  reviews  of  the  Digital  Production  System 
consistent  with  the  life-cycle  management  policies  and  procedures  contained  in  the  DoDD  8000- 
series.  Therefore,  a  Milestone  IV  review  following  the  Defense  Acquisition  Board  procedures  is 
unnecessary. 

Detailed  comments  on  the  report  findings  and  recommendations  are  enclosed.  We 
appreciate  the  opportunity  to  comment  on  the  draft  report 


Assistant  Secretary  of  Defense  (Conunand,  Control,  Conununications  and 
Intelligence)  Comments 


DoD  INSPECTOR  GENERAL  DRAFT  AUDIT  REPORT  -  DATED 
AUGUST  24,  1994 
(DOD  IG  PROJECT  NO.  3RC-8008) 

'^MANAGEMENT  OF  THE  DIGITAL  PRODUCTION 
SYSTEM  DEVELOPMENT  AT  THE 
DEFENSE  HAPPING  AGENCY" 


ITEM  SPECIFIC  COMMENTS 

Finding  A;  Management  of  the  Digital  Production  System 
Development 

Identifying  Customer  Requirements  for  thQ  PfgitaX  Pyp^wtipn 
System  (DPS)  .  The  report  stated  that  DMA  did  not  identify 
all  product  specifications  for  the  Flight  Information 
Publications  (FLIP)  and  Automated  Air  Facilities  Information 
File  (AAFIF)  products.  Specifically,  it  states  that  DMA  did 
not  develop  comprehensive  FLIP  and  AAFIF  product 
specifications  that  conformed  to  standards  and  were  formally 
reviewed.  Consequently,  18  customer-requested  specification 
changes  were  not  incorporated.  In  addition,  the  report 
faults  DMA  for  permitting  additional  changes  to  the  product 
specifications,  which  required  further  development. 

rnMMWKpr 2  Concur .  There  was  a  problem  with  omission  of 
several  changes  to  the  product  specifications;  however,  the 
IG  report  inaccurately  presents  the  product  specification 
control  process. 

The  Aeronautical  Data  Maintenance  (ADM)  subsystem  of  the  DPS 
supports  the  FLIP  and  AAFIF  products.  FLIP  and  AAFIF 
specifications,  which  are  developed  and  controlled  by 
external  organizations  (FLIP  Coordinating  Committee,  Federal 
Aviation  Agency,  International  Civil  Aeronautics 
Organization,  etc.)  change  about  every  3  months.  In 
accordance  with  a  Defense  Science  Board  recommendation,  the 
specifications  for  ADM  were  frozen  in  1989  to  provide  a 
stable  baseline  for  implementation.  An  ADM  baseline 
revision  was  planned  prior  to  full  operational  capability. 

In  order  to  meet  DMA's  mission  during  ADM  development,  the 
older  non-DPS  production  systems,  under  a  separate 
operational  configuration  management  process,  continued  to 
incorporate  product  specification  changes.  In  the  ADM 
baseline  revision  process,  18  of  150  changes  were  omitted. 

IG  RECQ^^gwHATiON  A,l;  Implement  procedures  to  identify  and 
coordinate  customer  requirements  for  FLIP  and  AAFIF  products 
before  specifications  are  given  to  the  contractor  for 
development . 


46 


Assistant  Secretary  of  Defense  (Command,  Control,  Communications  and 

Intelligence)  Comments 


roMMEMT;  Concur.  DMA  merged  all  AAFIF  and  FLIP  product 
specification  changes  into  one  configuration  management 
process.  By  November  1993/  all  FLIP  and  AAFIF  product 
specification  changes  were  baselined  to  the  ADM.  To  ensure 
timely  delivery  of  future  changes,  DMA  implemented  an  early 
coordination  process  with  the  contractor. 

The  revised  process  flow  is  documented  in  the  DMA 
Configuration  Policy  Handbook,  DMA  PH-907-1.  There  have 
been  no  further  problems  since  the  implementation  of  this 
new  process. 

DMA  is  also  working  with  the  external  organizations  to 
reduce  the  frequency  of  implementing  specification  changes 
from  three  to  six  months  updates.  This  would  greatly  reduce 
system  impacts  while  still  meeting  customer  needs  and  flight 
safety  requirements. 

DMA  considers  these  actions  to  have  effectively  implemented 
and  closed  IG  recommendation  A.l. 

Analyzing  Problems  in  DPS  Software  Development.  The  audit 
reported  that  engineers  could  not  track  development  problems 
due  to  inadequate  discrepancy  reporting  and  a  lack  of 
methodology  for  trend  analysis.  Specifically,  the  report 
stated  that  engineers  reviewing  Discrepancy  Reports  (DRs) , 
which  are  used  to  document  problems  with  the  DPS,  did  not 
enter  all  category  codes  in  19%  of  the  DRs  or  in  the 
Automated  Configuration  Management  System  (ACMS)  and  that 
lack  of  category  codes  information  made  trend  analysis 
impossible.  The  report  goes  on  to  state  that  only  19%  of 
the  82,000  predicted  software  defects  were  discovered  by 
production  startup  in  November  1992  compared  with  an 
expected  rate  of  70%  discovery  for  comparable  programs. 

COMMENTS  Partially  Concur.  The  DMA  Engineering  Review 
Boards  (ERBs)  use  a  variety  of  elements  to  track,  analyze, 
and  prioritize  development  problems  including  daily  and 
weekly  tri-center  reviews  with  production  personnel. 
Classification  by  category  as  directed  by  MIL  STD-2167B 
(i.e.,  hardware,  software,  documentation,  or  design  problem) 
is  provided  by  another  ACMS  field,  the  DR  TYPE,  which  has 
always  been  a  required  field.  The  category  code  further 
stratifies  software  problems  and  was  useful  in  identifying 
interface  problems  during  system  integration. 

During  the  DPS  development,  trend  analyses  were  performed, 
using  data  from  ACMS  as  well  as  the  development  contractor's 
configuration  control  systems.  These  analyses  were 
presented  by  senior  DMA  engineers  at  monthly  reviews  with 
the  Director,  DMA  Systems  Center  and  by  the  segment 
contractors  in  quarterly  reviews.  DMA  believes  the 
contractor  trend  analyses  mitigated  subsequent  DR  production 
impacts.  There  were,  however,  no  written  procedures  for 
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performing  these  analyses.  As  the  system  transitioned  into 
production,  the  emphasis  on  trend  analysis  was  reduced. 

During  segment  level  testing,  DMA  analysis  of  the  DPS 
software  indicated  that,  prior  to  delivery  to  site,  defects 
were  detected  and  removed  from  the  system  at  rates 
comparable  to  the  industry  average,  i.e.  70%  of  the  defects 
present  after  initial  coding  had  been  removed  prior  to 
delivery  to  system-level  test.  This  rate  compares  favorably 
to  comparable  DoD  test  programs  in  a  1989  OSD  Comptroller 
review, 

DMA  Systems  Center  completed  development  testing  of  system 
interfaces  and  major  functional  threads  (i.e.,  data 
extraction,  etc.)  in  October  1992.  Exercise  and  Rehearsal 
(E&R)  testing  of  the  major  functional  threads  using  users 
began  in  early  1992.  These  functional  threads  were  then 
scheduled  to  be  turned  over  to  production  incrementally.  In 
August  1993,  a  conscious  decision  was  made  by  DMA  to 
terminate  the  E&R  testing  prior  to  completion  of  all  threads 
in  favor  of  a  complete  turnover  to  production.  This  was 
based  on  an  assessment  that  continued  E&R  testing  provided 
only  marginal  value  when  compared  with  its  costs.  The 
decision  was  influenced  by  the  July  1993  "Advisory  Panel 
Report  on  DPS  Maintenance  Management"  which  recommended  that 
DMA  “direct  the  most  expeditious  introduction  of  DPS  into 
production  ...  to  achieve  the  earliest  return  on  its  large 
investment  and  for  discovery  and  resolution  of  inherent 
defects . " 

Since  delivery  to  site,  as  noted  in  the  IG  report,  the 
defect  discovery  rate  has  been  well  below  expected  values. 
DMA  believes  that  this  is  a  function  of  system  use,  driven 
by  the  long  pipeline  times  required  to  complete  a  production 
job  (e.g.,  six  months  for  a  new  1 : 50 , 000-scale  Topographic 
Line  Map) , 

TG  RECQMMEMDAT.lpff-A.^2;  Revise  procedures  for  discrepancy 
reporting  to  require  that  category  codes  be  entered  in 
discrepancy  reports  and  in  ACMS. 

COMMENT g  Concur .  ACMS  software  has  been  modified  to 
require  completion  of  the  category  code  field.  DMA 
considers  this  action  to  have  effectively  implemented  and 
closed  IG  recommendation  A. 2. 

IG  RECOMMENDATION  A. 3;  Establish  procedures  and  a 
methodology  for  analyzing  causes  of  and  trends  in  problems 
identified  in  discrepancy  reports. 

COMMENT !  Concur .  The  IG's  recommendation  to  formalize 
procedures  for  trend  analysis  and  provide  appropriate 
education  to  ERB  staffs  reflects  several  on-going  DMA 
initiatives . 
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DMA  is  presently  revamping  its  overall  DR  analysis  strategy 
and  expects  trend  analysis  to  play  an  important  role.  In 
December  1993,  DMA  issued  procedures  for  DPS  Engineering 
Review  Boards  and  production  components  to  institute  a 
metrics  data  collection  and  measurement  analysis  program. 

DMA  has  instituted  other  activities  to  improve  software 
quality.  DMA  is  currently  performing  static  analysis  of  the 
DPS  software  to  identify  overly  complex  modules.  The 
initial  analysis  will  be  completed  January  1995/  the 
results,  in  conjunction  with  actual  DR  trend  data,  will  be 
used  to  identify  software  units  that  are  candidates  for 
redesign. 

In  J\me  1994,  DMA  approved  the  Software  Measurement  and 
Metric  Analysis  Handbook  which  defines  the  life  cycle 
measurement  and  metric  analysis  processes  to  be  used  for 
DMA-designated  migration  systems.  Collection  and  analysis  of 
a  core  set  of  metrics  are  required  for  follow-on  DPS 
operations,  maintenance,  and  development  contracts. 

Based  on  these  initiatives,  formal  training  will  be 
developed  and  provided  to  the  ERB  staff  by  February  1995. 
With  the  efforts  outlined  above,  DMA  will  consider  IG 
recommendation  A,  3  closed  with  the  implementation  of  the  ERB 
training  in  February  1995. 

CPngiguration  Management  for  the  DPS.  The  IG  reported  that 
DMA  did  not  maintain  configuration  control  of  the  DPS 
because  it  had  not  resolved  9  of  11  Configuration  Management 
(CM)  deficiencies  identified  in  a  1993  DMA  internal  review. 
These  included  CM  training,  control  records,  and  CM 
guidelines  for  the  engineering  review  boards. 

COMMENT;  Partially  Concur.  Configuration  control  of  the 
DPS  has  been  maintained  throughout  the  life  cycle  of  the 
program  in  accordance  with  approved  CM  documents  including 
DPS  System  Configuration  Management  Plan,  System  Transition 
Integration  Plan,  and  the  DPS  Program  Implementation 
Directive.  The  referenced  deficiencies  did  exist  but  did 
not  result  in  loss  of  system  configuration  control. 

IG  Recommendation  A.  4;  Assign  personnel  to  the 
configuration  management  organization  responsible  for 
developing  configuration  management  procedures  and  for 
oversight  of  the  configuration  management  function. 

COMMENT;  Concur.  In  March  1994,  DMA  Systems  Center 
established  the  System  Test  and  Configuration  Management 
Division  within  the  Program  Execution  Directorate,  This 
division  has  now  been  staffed,  with  two  additional  full  time 
individuals  to  be  assigned  in  December  1994.  DMA  will 
consider  IG  recommendation  A. 4  closed  with  the  assignment  of 
these  individuals  in  December  1994. 
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IG  RECOMMENDATION  A.Si  Develop  and  implement  plans  with 
completion  dates  for  the  correction  of  the  deficiencies 
identified  in  the  DMA  Systems  Center  1993  Configuration 
Management  Review. 

COMMENT:  Concur,  The  remaining  deficiencies  will  be 
corrected,  except  one,  with  the  implementation  of  DMA  Policy 
Instruction  907'-!,  issued  January  1994  and  DMA  Procedural 
Handbook  (DMAPH)  907-1  which  was  approved  and  implemented 
at  the  June  1994  Executive  Configuration  Control  Board. 

The  remaining  deficiency,  "Level  of  CM  knowledge  and 
experience  is  inadequate,'  is  scheduled  to  be  closed  in 
December  1994.  DMA  Human  Resources  is  working  with  the 
Configuration  Management  Steering  Group  to  prepare  a  CM 
training  course,  schedule,  and  required  attendance  roster 
focused  on  implementation  of  procedures  identified  above. 

DMA  will  consider  IG  recommendation  A.  5  closed  with  the 
completion  of  this  training  in  December  1994. 

Proaram^naaement  Plan  for  pps  Modifications.  The  IG 
report  stated  that  DMA  had  not  developed  a  program 
mcuiagement  plan  that  clearly  defined  the  objectives  and 
development  strategy  for  the  short-  and  long-term 
modifications  planned  for  the  DPS.  It  also  stated  that  DMA 
plcinned  to  spend  $185  million  in  RDT&E  for  modifying  DPS  or 
developing  new  products  and  $220  million  on  research  and 
development  efforts  that  may  directly  or  indirectly  result 
in  modifications  to  DPS  operations  or  products. 

COMMENT:  Partially  Concur.  DMA  managed  the  DPS  development 

according  to  a  very  comprehensive  program  management  plan 
with  accompanying  system  engineering,  test  and  evaluation, 
and  segment  logistics  support  plans.  As  the  DPS 
transitioned  into  the  operational  DMA  production  system 
footprint,  further  modifications  were  planned  and  executed 
along  functional  lines  which  reflected  the  production  line 
environment  (i.e.,  Computer  Systems,  Production  Management, 
Product  Generation,  Hydrographic  Programs,  etc.).  In 
essence,  the  DPS  was  losing  its  identity  to  the  larger  MC&G 
production  System  (MPS) .  In  Oct  93,  DMA  formed  a  Technology 
and  Information  Steering  Committee  (TISC)  to  plan  the 
strategic  direction  for  this  MC&G  Production  System.  In  Feb 
1994,  the  MPS  was  defined  and  certified  as  a  migration 
system  in  accordance  with  ASD(C3I)  policy.  In  Aug  1994,  the 
TISC  produced  the  Technology  and  Information  Strategic  Plan 
(TISP)  which  details  strategic  goals  and  objectives  for  the 
MPS  with  a  supporting  MPS  migration  plan  which  details  the 
technical  road  map  for  achieving  those  goals.  In  response, 
DMA  Systems  Center's  functional  program  managers  are 
preparing  Program  Management  Plans  for  each  of  their  areas 
detailing  how  and  when  their  respective  modifications  will 
be  executed.  These  plans  are  due  December  1994. 
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IQ  RECQMWTgT^TiA^TiON  A,  6?  Develop  a  comprehensive  program 
management  plan  that  defines  the  modifications  to  be  made  to 
the  DPS  and  that  details  how  and  when  the  modifications  will 
be  accomplished. 

t  Concur .  In  May  1993,  DMA  decided  to  request 
approval  to  recapitalize  the  DPS  hardware  and  software  to 
dramatically  reduce  its  operations  and  maintenance  costs  and 
meet  changing  user  requirements  as  a  result  of  the  Cold  War 
cessation.  This  DPS  recapitalization  program  does  indeed 
necessitate  a  comprehensive  strategic  plan  which  is 
scheduled  to  be  prepared  by  February  1995. 

Finding  B:  Overaicht  of  the  Digital  Proquctipn  gygt^ 
l^velorwnant .  The  DoD  IG  reported  that  the  development  and 
modification  of  the  Digital  Production  System  (DPS)  lacked 
sufficient  oversight  and  that  because  DPS  was  not  subject  to 
acquisition  policies  in  DoD  directives  and  instructions,  the 
Defense  Acquisition  Board  had  not  reviewed  the  development  of 
the  program 

The  DoD  IG  also  noted  that  because  non- DoD  channels  and 
practices  were  used  in  the  development  of  DPS,  the  ASD(C3I)  did 
not  require  the  DMA  to  comply  with  acquisition  policies  in  DoD 
directives  and  instructions,  and  as  a  result,  the  Major 
Automated  Information  System  Review  Council  exempted  the  DPS 
from  review.  The  DoD  IG  noted  that  it  recommended  based  on  a 
1989  inspection  of  the  Defense  Mapping  Agency  that  the  Under 
Secretary  of  Defense  for  Acquisition  (now  the  Under  Secretary  of 
Defense  for  Acquisition  and  Technology)  conduct  a  Milestone  II 
review  of  the  program.  In  response,  the  Under  Secretary  of 
Defense  for  Acquisition  declined  to  review  the  program,  because 
the  Defense  Acquisition  Board  review  process  was  inappropriate 
for  reviewing  the  program's  development.  The  DoD  IG  also  noted 
that  external  oversight  of  DPS  was  provided  by  the  ASD(C3I) 
through  a  series  of  periodic  status  briefings  regarding  DPS 
development.  The  DoD  IG  also  found  that  the  DMA  personnel 
provided  additional  briefings  on  DPS  development  to 
congressional  committee  staff  with  a  demonstrated  need  to  know. 
In  addition,  DMA  convened  independent  panels  in  FY  1988  and 
FY  1993,  respectively  to  review  and  assess  DPS  development. 

The  DoD  IG  noted  that  effective  oversight  of  DPS  is  critical  to 
ensure  the  best  use  of  allocated  resources.  The  DoD  IG  further 
noted  that  DPS  is  an  automated  information  system  that  would 
have  been  reviewed  by  the  Major  Automated  Information  System 
Review  Council  (MAISRC) ,  provided  DPS  had  not  been  exempted  from 
review. 

IG  RECOMMENDATIONS  B.l;  Partially  Coag-U^.  The  DoD  disagrees 
with  the  DoD  IG  statement  that  DPS  was  not  subject  to  or 
required  to  comply  with  acquisition  policies.  As  DPS  was 
formerly  designated  by  the  Secretary  of  Defense  as  a  highly 
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sensitive,  classified  program  in  1982,  it  was  precluded  from  the 
traditional  life-cycle  management  reviews  under  MAI SRC  process 
or  DAB  process.  However,  consistent  with  the  authorities  and 
responsibilities  contained  in  DoD  Directive  5137.1  and  DoD 
Instruction  5000.2,  Part  2,  the  ASD(C3I)  conducted  oversight  of 
DPS  to  assess  development  progress  and  actions  to  complete  core 
program  activities.  However,  we  agree  that  completion  of  core 
capabilities  and  planned  product  improvements  warrant  continued 
oversight  review.  Since  development  oversight  currently  resides 
with  the  ASD(C3I),  we  do  not  agree  that  the  USD(A&T)  should 
conduct  a  Milestone  IV  review  of  the  planned  modifications  which 
are  valued  below  the  major  program  level.  Instead,  the  ASD(C3I) 
has  tasked  the  DASD{C3IA)  to  conduct  an  OSD  In-Process  Review  of 
the  Digital  Production  System  within  the  next  120  days.  The 
purpose  of  the  review  will  be  to  assess  the  program  status,  to 
review  the  DPS  Strategic  Plan,  and  to  determine  future  in- 
process  review  intervals. 

TG  RgCOMMENDATIONS  B.2:  PARTIALLY  CONCUR.  The  DoD  disagrees 
with  the  DoD  IG  Statement  that  DPS  was  not  subject  to 
independent  oversight  by  the  Assistant  Secretary  of  Defense 
(Command,  Control,  Communications  and  Intelligence) .  As  the 
program  was  formerly  designated  by  the  Secretary  of  Defense  as  a 
highly  sensitive,  classified  program  in  1982,  DPS  core 
activities  were  managed  by  the  ASD(C3I)  in  accordance  with  DoD 
Instructions  5000.2,  Part  2.  As  the  program  is  evolving  to  meet 
near-term  cold  war  requirements  through  a  series  of  DPS 
enhancements,  a  DPS  Strategic  Plan  will  be  developed  to 
establish  a  system-wide  baseline  so  that  future  focused 
oversight  reviews  of  the  progrcim  may  be  conducted  as  DPS 
migrates  to  full  and  open  competition  and  to  an  open  system 
environment . 
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